Embulk is an Open-source Pluggable Bulk Data Loaderto/from varieties of storages, file formats, databases, cloud services, and else.
Latest Versions
- Stable
- v0.11.0
- Note that v0.11 is not fully compatible with v0.9. Several plugins for v0.9 would not work with v0.11.
- Take a look at this article for details.
- Past stable
- v0.9.25
Recent Articles
Embulk is a bulk data loader. It helps data transfer between types of databases, storages, file formats, cloud services, and else.
Embulk supports:
- Combination of input and output from varieties of plugins
- Plugins released in Maven and Ruby gem repositories
- Automated guess of input file formats
- Parallel execution to deal with big data sets
- Transaction control to guarantee all-or-nothing
Community
- For Users
- User Forum: GitHub Discussions
- For Developers
- Chat: Zulip at https://embulk-dev.zulipchat.com/
- Developer Resources: https://dev.embulk.org/
Quick Start
Embulk runs on Java. Make sure Java Runtime is installed.
Embulk v0.9 (past stable) run only on Java 8.
Embulk v0.11 officially supports only Java 8, but expected to work somehow with Java 11, and 17. We are going to test v0.11 with Java 11 and 17 so that we can declare an official support for them.
Linux & macOS
curl --create-dirs -o ~/.embulk/bin/embulk -L "https://dl.embulk.org/embulk-latest.jar"
chmod +x ~/.embulk/bin/embulk
echo 'export PATH="$HOME/.embulk/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
Windows
PowerShell -Command "& {[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::TLS12; Invoke-WebRequest http://dl.embulk.org/embulk-latest.jar -OutFile embulk.bat}"
Set up with JRuby
Download your preferred version 9.x.y of JRuby Complete jar from JRuby Downloads, and put it in your favorite directory.
Embulk is tested with old JRuby 9.1.15.0 as of now. (However, older JRuby has an issue in installing Ruby gems. You may need to try newer JRuby.)
Then, create ~/.embulk/embulk.properties
as follows.
jruby=file:///path/to/jruby-complete-9.x.y.z.jar
Finally, install some Ruby gems.
embulk gem install embulk -v 0.11.0 # Must be the same Embulk version
embulk gem install msgpack
embulk gem install bundler # if you need Bundler
embulk gem install liquid # if you need Liquid
Next steps
The embulk example
command generates an example CSV file.
You can try Embulk quickly by:
embulk example ./try1
embulk guess ./try1/seed.yml -o config.yml
embulk preview config.yml
embulk run config.yml