data-file-reader

A library which reads and writes table-like streams of rows from common file formats.

The goal of this project is to make the reading and writing of table style file formats simple. If you have specific file format needs, you may review currently supported file formats using the features dropdown in the top right, or or write your own.

The base file encoder and decoder both use the ServiceLocator to detect available converters. Simply register any new mime type adapters in the corresponding META-INF/services file, and they will be automatically applied based on the mimetype you select.

The below dependency includes csv, bson, and json support. You may also include the individual file converters individually.

<dependency>
    <groupId>net.krotscheck.dfr</groupId>
    <artifactId>data-file-reader-all</artifactId>
    <version>1.0.10</version>
</dependency>

Reading a File

To read a file, open an InputStream from your location, construct a decoder, and treat it like an iterator.

InputStream inputStream = ResourceUtil.getResourceAsStream("my_file");
String myMimeType = "text/mymimetype";
FileStreamDecoder decoder = new FileStreamDecoder(inputStream, myMimeType);
decoder.setInputStream(inputStream);

for(Map<String, Object> row : decoder) {
    // Do something
}
decoder.close();

Writing a File

You may write arbitrary data rows to a file format of your choosing, simply by specifying an outputStream and a mimetype.

OutputStream outputStream = new FileOutputStream("my_file_path");
String myMimeType = "text/mymimetype";
FileStreamEncoder encoder = new FileStreamEncoder(outputStream, myMimeType);

Map<String, Object> myRow = new LinkedHashMap<String, Object>();
encoder.write(myRow);

encoder.close();

Converting a File

This library also provides a convenience file format converter. You may either call run() directly, or you may pass it to a thread as shown below.

    IDataDecoder decoder = new MyDataDecoder();
    IDataEncoder encoder = new MyDataEncoder();

    FileStreamConverter fsc = new FileStreamConverter(decoder, encoder);

    Thread converterThread = new Thread(fsc);
    converterThread.start();

Filtering a File

If you would like to apply filters to your data as it’s in flight, you may apply filters to either your decoder, encoder, or converter. These should implement the IDataFilter interface, and, as a best practice, should not perform a destructive change.

    IDataDecoder decoder = new MyDataDecoder();
    IDataEncoder encoder = new MyDataEncoder();
    IDataFilter filter = new MyDataFilter();

    FileStreamConverter fsc = new FileStreamConverter(decoder, encoder);

    // You may apply your filter to any component of your filtering stream.
    decoder.addFilter(filter);
    // ... or ...
    encoder.addFilter(filter);
    // ... or ...
    fsc.addFilter(filter);

    Thread converterThread = new Thread(fsc);
    converterThread.start();