Skip to content

Latest commit

 

History

History
71 lines (45 loc) · 2.62 KB

README.md

File metadata and controls

71 lines (45 loc) · 2.62 KB
Raw

Utility

This repository contains a Go CLI that performs file merging operations. The CLI merges all JSONL files in a specified directory into a single JSONL file and stores it in an output directory. Additionally, if the --format csv flag is used, the output is a CSV file.

Task Description

Objective: Create a Go CLI tool named util that performs file merging operations. The tool should merge all JSONL files in a specified directory into a single JSONL file and store it in an output directory. Additionally, if the --format csv flag is used, the output should be a CSV file.

Requirements

  1. CLI Flags:

    • --operation merge: Specifies the operation to be performed.
    • --input-path ./<any_path>: Path to the directory containing JSONL files to be merged.
    • --output-path ./<any_path>: Path to the directory where the merged file will be stored.
    • --format csv: (Optional) If provided, the output should be a CSV file instead of a JSONL file.

  2. Functionality:

    • Read all JSONL files from the input directory.
    • Merge the content of all JSONL files into a single JSONL file.
    • If the --format csv flag is detected, convert the merged JSONL content to CSV format.
    • Save the merged file (in JSONL or CSV format) to the specified output directory.

  3. Bonus: Implement the --format csv flag to output a CSV file.

Steps to Complete the Task

  1. Setup Go Environment:

    • Ensure Go is installed on your machine.
    • Create a new Go module.

  2. Implement the CLI Tool:

    • Use a CLI package like flag or cobra for parsing command-line arguments.
    • Implement functions to read and merge JSONL files.
    • Implement functions to convert JSONL to CSV format (if required).
    • Implement the main function to handle CLI arguments and execute the appropriate functions.

  3. Testing:

    • Create a few sample JSONL files for testing.
    • Test the tool by running the command with different flag combinations.

  4. Documentation:

    • Provide clear instructions on how to use the tool, including examples.

Example Commands

  1. Merge JSONL files into a single JSONL file:

    ./bin/util --operation merge --input-path ./data/jsonl_files/ --output-path ./output_data

  2. Merge JSONL files and output as a CSV file:

    ./bin/util --operation merge --input-path ./data/jsonl_files/ --output-path ./output_data --format csv

Submission

Submit the following:

  1. The complete Go code.
  2. Sample input JSONL files for testing.
  3. A README file with instructions on how to build and run the tool, including example commands.

Good luck with the task!