Skip to content
This repository has been archived by the owner on Apr 4, 2019. It is now read-only.

Update Readme markdown syntax for github #57

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ _Forked from the awesome [kafka-connect-hdfs](https://github.com/confluentinc/ka

StreamX is a kafka-connect based connector to copy data from Kafka to Object Stores like Amazon s3, Google Cloud Storage and Azure Blob Store. It focusses on reliable and scalable data copying. It can write the data out in different formats (like parquet, so that it can readily be used by analytical tools) and also in different partitioning requirements.

##Features :
## Features :

StreamX inherits rich set of features from kafka-connect-hdfs.
- Support for writing data in Avro and Parquet formats.
Expand All @@ -18,7 +18,7 @@ In addition to these, we have made changes to the following to make it work effi
- Exactly-once guarantee using WAL
- Support for storing Hive tables in Qubole's hive metastore (coming soon)

##Getting Started:
## Getting Started:

Pre-req : StreamX is based on Kafka Connect framework, which is part of Kafka project. Kafka Connect is added in Kafka 0.9, hence StreamX can only be used with Kafka version >= 0.9. To download Kafka binaries, check [here](http://kafka.apache.org/downloads.html).

Expand Down Expand Up @@ -137,7 +137,7 @@ docker run -d -p 8083:8083 --env CONNECT_BOOTSTRAP_SERVERS=public_dns:9092 --env

```

##Roadmap
## Roadmap
- Support other object stores like Google Cloud Storage and Azure Blob Store
- Currently, data can be written in avro/parquet format. This project will add support for more formats
- Deal with features related to s3, like small-file consolidation