Embulk is an Open-source Pluggable Bulk Data Loaderto/from varieties of storages, file formats, databases, cloud services, and else.
Latest Versions
- Stable
- v0.11.4
- Note that v0.11 is not fully compatible with v0.9. Several plugins for v0.9 would not work with v0.11.
- Take a look at this article for details.
- Past stable
- v0.9.25
Recent Articles
Embulk is a bulk data loader. It helps data transfer between types of databases, storages, file formats, cloud services, and else.
Embulk supports:
- Combination of input and output from varieties of plugins
- Plugins released in Maven and Ruby gem repositories
- Automated guess of input file formats
- Parallel execution to deal with big data sets
- Transaction control to guarantee all-or-nothing
Community
- For Users
- User Forum: GitHub Discussions
- For Developers
- Chat: Zulip at https://embulk-dev.zulipchat.com/
- Developer Resources: https://dev.embulk.org/
Quick Start
Set up the Java Runtime Environment. Embulk runs on Java. Embulk v0.11 officially supports only Java 8, but expected to work somehow with Java 11, 17, and 21.
Download the latest `embulk-X.Y.Z.jar` from: [GitHub Releases](https://github.com/embulk/embulk/releases)
Try the following commands. embulk example
command generates an example CSV file.
java -jar embulk-X.Y.Z.jar example ./try1
java -jar embulk-X.Y.Z.jar guess ./try1/seed.yml -o config.yml
java -jar embulk-X.Y.Z.jar preview config.yml
java -jar embulk-X.Y.Z.jar run config.yml
Install an Embulk plugin
Install an Embulk plugin (ex. S3 Input Plugin) with the following command.
java -jar embulk-X.Y.Z.jar install "org.embulk:embulk-input-s3:0.6.0"
Set up with JRuby
If you need to use a RubyGems-style Embulk plugin, you'll have to set up with JRuby.
Download your preferred version 9.x.y of JRuby Complete jar from JRuby Downloads, and put it in your favorite directory.
Embulk is tested with old JRuby 9.1.15.0 as of now. (However, older JRuby has an issue in installing Ruby gems. You may need to try newer JRuby.)
Then, create ~/.embulk/embulk.properties
as follows.
jruby=file:///path/to/jruby-complete-9.x.y.z.jar
Finally, install some Ruby gems.
embulk gem install embulk -v 0.11.4 # Must be the same Embulk version
embulk gem install msgpack
embulk gem install bundler # if you need Bundler
embulk gem install liquid # if you need Liquid