From 96a54af78038298dacc21fa2837633b2ff097cca Mon Sep 17 00:00:00 2001 From: Sergey Prokhorov Date: Thu, 7 Dec 2023 13:22:54 +0100 Subject: [PATCH] Update README --- CHANGELOG.md | 16 +++----- README.md | 110 +++++++++++++++++++++++++++++++++++++-------------- 2 files changed, 85 insertions(+), 41 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0054628..6d4b318 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,19 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] - yyyy-mm-dd -## [0.0.2] - yyyy-mm-dd - -### Changed - -- A changelog is a file which contains a curated, chronologically ordered list of notable changes for each version of a project. - -## [0.0.1] - yyyy-mm-dd +## [0.0.1] - 2023-12-07 ### Added -- To make it easier for users and contributors to see precisely what notable changes have been made between each release (or version) of the project. +- The initial release: interface, set of helper functions, property-based tests, CI. -[unreleased]: https://github.com/klarna-incubator/TODO/compare/v1.1.0...HEAD -[0.0.2]: https://github.com/klarna-incubator/TODO/compare/v0.0.1...v0.0.2 -[0.0.1]: https://github.com/klarna-incubator/TODO/releases/tag/v0.0.1 +[unreleased]: https://github.com/klarna-incubator/iterator-erl/compare/v1.1.0...HEAD + +[0.0.1]: https://github.com/klarna-incubator/iterator-erl/releases/tag/v0.0.1 diff --git a/README.md b/README.md index af361c7..7150336 100644 --- a/README.md +++ b/README.md @@ -1,51 +1,101 @@ -# Project Name -> Short blurb about what your project does. +# iterator.erl +> Lazy sequences simulating stdlib `lists` module API -[![Build Status][ci-image]][ci-url] +![Build Status][https://github.com/klarna-incubator/iterator-erl/actions/workflows/ci.yml/badge.svg] [![License][license-image]][license-url] [![Developed at Klarna][klarna-image]][klarna-url] -One to two paragraph statement about your project and what it does. +This library provides an interface to define your own "lazy sequences" and also provides a list of +helpers to operate on such sequences that mimicks the OTP stdlib [lists](https://www.erlang.org/doc/man/lists) +functions. +"lazy sequences" allows you to operate on potentially huge streams of data the same way you would +operate on them if they are completely loaded into memory as a very long list, but doing so only +using constant memory footprint. It achieves that by loading one-list-item at a time. -## First steps +## Usage example -
- Installation (for Admins) - - Currently, new repositories can be created only by a Klarna Open Source community lead. Please reach out to us if you need assistance. - - 1. Create a new repository by clicking ‘Use this template’ button. - - 2. Make sure your newly created repository is private. - - 3. Enable Dependabot alerts in your candidate repo settings under Security & analysis. You need to enable ‘Allow GitHub to perform read-only analysis of this repository’ first. -
+It's primary intention is that you could define your own "iterators" using our generic interface +and then use our helper functions to build the "processing" pipeline for the stream of data. -1. Update `README.md` and `CHANGELOG.md`. +For example, let's implement an iterator that returns next line from a file each time: -2. Optionally, clone [the default contributing guide](https://github.com/klarna-incubator/.github/blob/main/CONTRIBUTING.md) into `.github/CONTRIBUTING.md`. +```erlang +file_line_iterator(FileName) -> + {ok, Fd} = file:open(Filename, [read, read_ahead, raw]), + iterator:new(fun yield_file_line/1, Fd, fun close_file/1). -3. Do *not* edit `LICENSE`. +yield_file_line(Fd) -> + case file:read_line(Fd) of + {ok, Line} -> + {Line, Fd}; + eof -> + done; + {error, Reason} -> + error(Reason) + end. -## Usage example +close_file(Fd) -> + file:close(Fd). +``` -A few motivating and useful examples of how your project can be used. Spice this up with code blocks and potentially more screenshots. +You use `iterator:new/3` to create the (opaque) iterator structure, where + +* 1st argument is the "yield" function - the function that returns either + `{NextSequenceElement, NewState}` or atom `done` when sequence is exhausted +* 2nd argument is the initial state of the iterator (in this example it does not change) +* 3rd optional argument is the "close" garbage-collection function that will be called when + iterator is exhausted or discarded (eg, used with `iterator:takewhile/2`) + +After that you can use the primitive `iterator:next/1` function that returns either +`{ok, NextElement, NewIterator}` or `done`. But the real power comes when you build +a "processing pipeline" instead. + +Then we may build a "processing pipeline" for this iterator. Let's say we want to filter-out the +lines that match a regular expression: + +```erlang +LinesIterator = file_line_iterator("my_file.txt"), +MatchingIterator = + iterator:filter( + fun(Line) -> + case re:run(Line, "^[0-9]+$") of + nomatch -> + false; + {match, _} -> + true + end + end, LinesIterator). +``` -_For more examples and usage, please refer to the [Docs](TODO)._ +And then we want to convert each matching line to integer -## Development setup +```erlang +IntegerIterator = iterator:map(fun erlang:binary_to_integer/1, MatchingIterator). +``` -Describe how to install all development dependencies and how to run an automated test-suite of some kind. Potentially do this for multiple platforms. +And finally we want to sum all the integers -```sh -make install -npm test +```erlang +Sum = iterator:fold(fun(Int, Acc) -> Int + Acc end, 0, IntegerIterator). ``` -## How to contribute +The `iterator:fold/2` is different from other pipeline functions because it does not return +the new iterator, but it "forces" the execution of iterator by reading inner iterator's elements +one-by-one and applying `fun` to them, maintaining the `Acc` state. +Another such functions are `iterator:to_list/1`, `iterator:search/2`, `iterator:mapfoldl/3`. -See our guide on [contributing](.github/CONTRIBUTING.md). +With this code, using `iterator`, we managed to go through the whole file never keeping more than +a single line in-memory but were able to work with it using the same code style and high-order +functions as what we would use if we read all the file lines in memory. + +## Setup + +Add it to your `rebar.config` + +```erlang +{deps, [iterator]}. +``` ## Release History @@ -53,7 +103,7 @@ See our [changelog](CHANGELOG.md). ## License -Copyright © 2022 Klarna Bank AB +Copyright © 2023 Klarna Bank AB For license details, see the [LICENSE](LICENSE) file in the root of this project.