Skip to content

Commit

Permalink
Fix gitbook edits + reorg changelogs (airbytehq#1513)
Browse files Browse the repository at this point in the history
  • Loading branch information
michel-tricot authored Jan 4, 2021
1 parent 302e3d6 commit 7d1a454
Show file tree
Hide file tree
Showing 16 changed files with 538 additions and 457 deletions.
8 changes: 4 additions & 4 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@
* [Config & Persistence](tutorials/airbyte-config-persistence.md)
* [Building a toy connector](tutorials/toy-connector.md)
* [Beginner's Guide to the AirbyteCatalog](tutorials/beginners-guide-to-catalog.md)
* [Changelog](changelog.md)
* [Releases](deploying-airbyte/releases.md)
* [Changelog](changelog/README.md)
* [Platform](changelog/platform.md)
* [Connectors](changelog/connectors.md)
* [Roadmap](roadmap.md)
* [Connectors](integrations/README.md)
* [Sources](integrations/sources/README.md)
Expand Down Expand Up @@ -54,7 +55,6 @@
* [Redshift](integrations/destinations/redshift.md)
* [Snowflake](integrations/destinations/snowflake.md)
* [Custom or New Connector](integrations/custom-connectors.md)
* [Connector Changelog](integrations/integrations-changelog.md)
* [Deploying Airbyte](deploying-airbyte/README.md)
* [On Your Workstation](deploying-airbyte/on-your-workstation.md)
* [On AWS \(EC2\)](deploying-airbyte/on-aws-ec2.md)
Expand All @@ -81,7 +81,7 @@
* [Updating Documentation](contributing-to-airbyte/updating-documentation.md)
* [Templates](contributing-to-airbyte/templates/README.md)
* [Connector Doc Template](contributing-to-airbyte/templates/integration-documentation-template.md)
* [Technical Support](technical-support/README.md)
* [Technical Support](technical-support.md)
* [Company Handbook](company-handbook/README.md)
* [Story](company-handbook/future-milestones.md)
* [Culture and Values](company-handbook/culture-and-values.md)
Expand Down
4 changes: 3 additions & 1 deletion docs/architecture/job-state-machine.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,6 @@
Jobs in the worker follow the following state machine.

![Job state machine](../.gitbook/assets/job-state-machine.png)
[Image Source](https://docs.google.com/drawings/d/1oMahOg1T8cssxiimV8u4lChbQP5D-wVrSjdMSgxdjiQ/edit?usp=sharing)

[Image Source](https://docs.google.com/drawings/d/1oMahOg1T8cssxiimV8u4lChbQP5D-wVrSjdMSgxdjiQ/edit)

80 changes: 23 additions & 57 deletions docs/changelog/README.md
Original file line number Diff line number Diff line change
@@ -1,80 +1,46 @@
---
description: Be sure to not miss out on new features and improvements!
---

# Changelog

This is the changelog for Airbyte core. For our connector changelog, please visit our [Connector Changelog](../integrations/integrations-changelog.md) page.
## Airbyte Core Releases

Airbyte Core is any non-connector code. This includes the scheduler, workers, api, web app, and the Airbyte protocol.

If you're interested in our progress on the Airbyte platform, please read below!
### Production v. Dev Releases

## 0.8.0 - delivered on 12/17/2020
The "production" version of Airbyte is the version of the app specified in `.env`. With each production release, we update the version in the `.env` file. This version will always be available for download on DockerHub. It is the version of the app that runs when a user runs `docker-compose up`.

* **Incremental - Append"**
* We now allow sources to replicate only new or modified data. This enables to avoid re-fetching data that you have already replicated from a source.
* The delta from a sync will be _appended_ to the existing data in the data warehouse.
* Here are [all the details of this feature](../architecture/incremental.md).
* It has been released for 15 connectors, including Postgres, MySQL, Intercom, Zendesk, Stripe, Twilio, Marketo, Shopify, GitHub, and all the destination connectors. We will expand it to all the connectors in the next couple of weeks.
* **Other features:**
* Improve interface for writing python sources \(should make writing new python sources easier and clearer\).
* Add support for running Standard Source Tests with files \(making them easy to run for any language a source is written in\)
* Add ability to reset data for a connection.
* **Bug fixes:**
* Update version of test containers we use to avoid pull issues while running tests.
* Fix issue where jobs were not sorted by created at in connection detail view.
* **New sources:** Intercom, Mixpanel, Jira Cloud, Zoom, Drift, Microsoft Teams
The "development" version of Airbyte is the head of master branch. It is the version of the app that runs when a user runs `docker-compose --env-file .env.dev -f docker-compose.yaml -f docker-compose.dev.yaml up`.

## 0.7.0 - delivered on 12/07/2020
### Production Release Schedule

* **New destination:** our own **Redshift** warehouse connector. You can also use this connector for Panoply.
* **New sources**: 8 additional source connectors including Recurly, Twilio, Freshdesk. Greenhouse, Redshift \(source\), Braintree, Slack, Zendesk Support
* Bug fixes
#### Scheduled Releases

## 0.6.0 - delivered on 11/23/2020
Airbyte currently releases a new minor version of the application on a weekly basis. Generally this weekly release happens on Monday or Tuesday.

* Support **multiple destinations**
* **New source:** Sendgrid
* Support **basic normalization**
* Bug fixes
#### Hotfixes

## 0.5.0 - delivered on 11/18/2020
Airbyte releases a new version whenever it discovers and fixes a bug that blocks any mission critical functionality.

* **New sources:** 10 additional source connectors, including Files \(CSV, HTML, JSON...\), Shopify, MSSQL, Mailchimp
**Mission Critical**

## 0.4.0 - delivered on 11/04/2020
e.g. Non-ASCII characters break the Salesforce source.

Here is what we are working on right now:
**Non-Mission Critical**

* **New destination**: our own **Snowflake** warehouse connector
* **New sources:** Facebook Ads, Google Ads.
e.g. Buttons in the UI are offset.

## 0.3.0 - delivered on 10/30/2020
#### Unscheduled Releases

* **New sources:** Salesforce, GitHub, Google Sheets, Google Analytics, Hubspot, Rest APIs, and MySQL
* Integration test suite for sources
* Improve build speed
We will often release more frequently than the weekly cadence if we complete a feature that we know that a user is waiting on.

## 0.2.0 - delivered on 10/21/2020
### Development Release Schedule

* **a new Admin section** to enable users to add their own connectors, in addition to upgrading the ones they currently use
* improve the developer experience \(DX\) for **contributing new connectors** with additional documentation and a connector protocol
* our own **BigQuery** warehouse connector
* our own **Postgres** warehouse connector
* simplify the process of supporting new Singer taps, ideally make it a 1-day process
As soon as a feature is on master, it is part of the development version of Airbyte. We merge features as soon as they are ready to go \(have been code reviewed and tested\). We attempt to keep the development version of the app working all the time. We are iterating quickly, however, and there may be intermittent periods where the development version is broken.

## 0.1.0 - delivered on 09/23/2020
If there is ever a feature that is only on the development version, and you need it on the production version, please let us know. We are very happy to do ad-hoc production releases if it unblocks a specific need for one of our users.

This is our very first release after 2 months of work.
## Airbyte Connector Releases

* **New sources:** Stripe, Postgres
* **New destinations:** BigQuery, Postgres
* **Only one destination**: we only support one destination in that 1st release, but you will soon be able to add as many as you need.
* **Logs & monitoring**: you can now see your detailed logs
* **Scheduler:** you now have 10 different frequency options for your recurring syncs
* **Deployment:** you can now deploy Airbyte via a simple Docker image, or directly on AWS and GCP
* **New website**: this is the day we launch our website - airbyte.io. Let us know what you think
* **New documentation:** this is the 1st day for our documentation too
* **New blog:** we published a few articles on our startup journey, but also about our vision to making data integrations a commodity.
Each connector is tracked with its own version. These versions are separate from the versions of Airbyte Core. We generally will bump the version of a connector anytime we make a change to it. We rely on a large suite of tests to make sure that these changes do not cause regressions in our connectors.

Stay tuned, we will have new sources and destinations very soon! Don't hesitate to subscribe to our [newsletter](https://airbyte.io/#subscribe-newsletter) to receive our product updates and community news.
When we updated the version of a connector, we usually update the connector's version in Airbyte Core as well. Keep in mind that you might not see the updated version of that connector in the production version of Airbyte Core until after a production release of Airbyte Core.

74 changes: 74 additions & 0 deletions docs/changelog/connectors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
description: Do not miss the new connectors we support!
---

# Connectors

**You can request new connectors directly** [**here**](https://github.com/airbytehq/airbyte/issues/new?assignees=&labels=area%2Fintegration%2C+new-integration&template=new-integration-request.md&title=)**.**

Note: Airbyte is not built on top of Singer, but is compatible with Singer's protocol. Airbyte's ambitions go beyond what Singer enables to do, so we are building our own protocol that will keep its compatibility with Singer's one.

## Currently under construction

**New sources:** Hive, MongoDB, Mailgun, Drift, Microsoft Teams
**New destinations:** Panoply, S3, MySQL, Kafka, MongoDB

## 12/10/2020

**New sources:** [Intercom](../integrations/sources/intercom.md), [Mixpanel](../integrations/sources/mixpanel.md), [Jira Cloud](../integrations/sources/jira.md), [Zoom](../integrations/sources/zoom.md)

## 12/07/2020

**New sources:** [Slack](../integrations/sources/slack.md), [Braintree](../integrations/sources/braintree.md), [Zendesk Support](../integrations/sources/zendesk-support.md)

## 12/04/2020

**New sources:** [Redshift](../integrations/sources/redshift.md), [Greenhouse](../integrations/sources/greenhouse.md)
**New destination:** [Redshift](../integrations/destinations/redshift.md)

## 11/30/2020

**New sources:** [Freshdesk](../integrations/sources/freshdesk.md), [Twilio](../integrations/sources/twilio.md)

## 11/25/2020

**New source:** [Recurly](../integrations/sources/recurly.md)

## 11/23/2020

**New source:** [Sendgrid](../integrations/sources/sendgrid.md)

## 11/18/2020

**New source:** [Mailchimp](../integrations/sources/mailchimp.md)

## 11/13/2020

**New source:** [MSSQL](../integrations/sources/mssql.md)

## 11/11/2020

**New source:** [Shopify](../integrations/sources/shopify.md)

## 11/09/2020

**New sources:** [Files \(CSV, JSON, HTML...\)](../integrations/sources/file.md)

## 11/04/2020

**New sources:** [Facebook Ads](../integrations/sources/facebook-marketing-api.md), [Google Ads](../integrations/sources/google-adwords.md), [Marketo](../integrations/sources/marketo.md)
**New destination:** [Snowflake](../integrations/destinations/snowflake.md)

## 10/30/2020

**New sources:** [Salesforce](../integrations/sources/salesforce.md), [Google Analytics](../integrations/sources/googleanalytics.md), [Hubspot](../integrations/sources/hubspot.md), [GitHub](../integrations/sources/github.md), [Google Sheets](../integrations/sources/google-sheets.md), [Rest APIs](connectors.md), and [MySQL](../integrations/sources/mysql.md)

## 10/21/2020

**New destinations:** we built our own connectors for [BigQuery](../integrations/destinations/bigquery.md) and [Postgres](../integrations/destinations/postgres.md), to ensure they are of the highest quality.

## 09/23/2020

**New sources:** [Stripe](../integrations/sources/stripe.md), [Postgres](../integrations/sources/postgres.md)
**New destinations:** [BigQuery](../integrations/destinations/bigquery.md), [Postgres](../integrations/destinations/postgres.md), [local CSV](../integrations/destinations/local-csv.md)

80 changes: 80 additions & 0 deletions docs/changelog/platform.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
---
description: Be sure to not miss out on new features and improvements!
---

# Platform

This is the changelog for Airbyte core. For our connector changelog, please visit our [Connector Changelog](connectors.md) page.

If you're interested in our progress on the Airbyte platform, please read below!

## 0.8.0 - delivered on 12/17/2020

* **Incremental - Append"**
* We now allow sources to replicate only new or modified data. This enables to avoid re-fetching data that you have already replicated from a source.
* The delta from a sync will be _appended_ to the existing data in the data warehouse.
* Here are [all the details of this feature](../architecture/incremental.md).
* It has been released for 15 connectors, including Postgres, MySQL, Intercom, Zendesk, Stripe, Twilio, Marketo, Shopify, GitHub, and all the destination connectors. We will expand it to all the connectors in the next couple of weeks.
* **Other features:**
* Improve interface for writing python sources \(should make writing new python sources easier and clearer\).
* Add support for running Standard Source Tests with files \(making them easy to run for any language a source is written in\)
* Add ability to reset data for a connection.
* **Bug fixes:**
* Update version of test containers we use to avoid pull issues while running tests.
* Fix issue where jobs were not sorted by created at in connection detail view.
* **New sources:** Intercom, Mixpanel, Jira Cloud, Zoom, Drift, Microsoft Teams

## 0.7.0 - delivered on 12/07/2020

* **New destination:** our own **Redshift** warehouse connector. You can also use this connector for Panoply.
* **New sources**: 8 additional source connectors including Recurly, Twilio, Freshdesk. Greenhouse, Redshift \(source\), Braintree, Slack, Zendesk Support
* Bug fixes

## 0.6.0 - delivered on 11/23/2020

* Support **multiple destinations**
* **New source:** Sendgrid
* Support **basic normalization**
* Bug fixes

## 0.5.0 - delivered on 11/18/2020

* **New sources:** 10 additional source connectors, including Files \(CSV, HTML, JSON...\), Shopify, MSSQL, Mailchimp

## 0.4.0 - delivered on 11/04/2020

Here is what we are working on right now:

* **New destination**: our own **Snowflake** warehouse connector
* **New sources:** Facebook Ads, Google Ads.

## 0.3.0 - delivered on 10/30/2020

* **New sources:** Salesforce, GitHub, Google Sheets, Google Analytics, Hubspot, Rest APIs, and MySQL
* Integration test suite for sources
* Improve build speed

## 0.2.0 - delivered on 10/21/2020

* **a new Admin section** to enable users to add their own connectors, in addition to upgrading the ones they currently use
* improve the developer experience \(DX\) for **contributing new connectors** with additional documentation and a connector protocol
* our own **BigQuery** warehouse connector
* our own **Postgres** warehouse connector
* simplify the process of supporting new Singer taps, ideally make it a 1-day process

## 0.1.0 - delivered on 09/23/2020

This is our very first release after 2 months of work.

* **New sources:** Stripe, Postgres
* **New destinations:** BigQuery, Postgres
* **Only one destination**: we only support one destination in that 1st release, but you will soon be able to add as many as you need.
* **Logs & monitoring**: you can now see your detailed logs
* **Scheduler:** you now have 10 different frequency options for your recurring syncs
* **Deployment:** you can now deploy Airbyte via a simple Docker image, or directly on AWS and GCP
* **New website**: this is the day we launch our website - airbyte.io. Let us know what you think
* **New documentation:** this is the 1st day for our documentation too
* **New blog:** we published a few articles on our startup journey, but also about our vision to making data integrations a commodity.

Stay tuned, we will have new sources and destinations very soon! Don't hesitate to subscribe to our [newsletter](https://airbyte.io/#subscribe-newsletter) to receive our product updates and community news.

46 changes: 0 additions & 46 deletions docs/changelog/releases.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/company-handbook/team.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,6 @@ Charles was a senior backend engineer at rideOS. Previous to that, he was a seni

## **Chris Duong**

Chris is a data veteran with experience in the Silicon Valley, Spain, Germany and France. He was the head of data of Comet and responsible for growing their data team in addition to building their infrastructure.
Chris is a data veteran with experience in the Silicon Valley, Spain, Germany and France. He was the head of data of Comet and responsible for growing their data team in addition to building their infrastructure.
[**LinkedIn**](https://www.linkedin.com/in/christopheduong/) **\|** [**Twitter**](https://twitter.com/ChristopheDuong)

Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,7 @@ npm run generate

and choose the relevant template. This will generate a new connector in the `airbyte-integrations/connectors/<your-connector>` directory.

Follow the instructions generated in the `NEW_SOURCE_CHECKLIST.md` checklist to complete the connector.
The checklist contains helpful tips about how to implement your connector
Follow the instructions generated in the `NEW_SOURCE_CHECKLIST.md` checklist to complete the connector. The checklist contains helpful tips about how to implement your connector

### 2. Integration tests

Expand Down
Loading

0 comments on commit 7d1a454

Please sign in to comment.