Skip to content

Commit

Permalink
feat: Über Migration (and Boxo rename)
Browse files Browse the repository at this point in the history 
Include rename from:
  github.com/ipfs/go-libipfs => github.com/ipfs/boxo

This unmigrate:
  ./blocks => github.com/ipfs/go-block-format

Migrated repos:
- github.com/ipfs/interface-go-ipfs-core         => ./coreiface
- github.com/ipfs/go-pinning-service-http-client => ./pinning/remote/client
- github.com/ipfs/go-path                        => ./path
- github.com/ipfs/go-namesys                     => ./namesys
- github.com/ipfs/go-mfs                         => ./mfs
- github.com/ipfs/go-ipfs-provider               => ./provider
- github.com/ipfs/go-ipfs-pinner                 => ./pinning/pinner
- github.com/ipfs/go-ipfs-keystore               => ./keystore
- github.com/ipfs/go-filestore                   => ./filestore
- github.com/ipfs/go-ipns                        => ./ipns
- github.com/ipfs/go-blockservice                => ./blockservice
- github.com/ipfs/go-ipfs-chunker                => ./chunker
- github.com/ipfs/go-fetcher                     => ./fetcher
- github.com/ipfs/go-ipfs-blockstore             => ./blockstore
- github.com/ipfs/go-ipfs-posinfo                => ./filestore/posinfo
- github.com/ipfs/go-ipfs-util                   => ./util
- github.com/ipfs/go-ipfs-ds-help                => ./datastore/dshelp
- github.com/ipfs/go-verifcid                    => ./verifcid
- github.com/ipfs/go-ipfs-exchange-offline       => ./exchange/offline
- github.com/ipfs/go-ipfs-routing                => ./routing
- github.com/ipfs/go-ipfs-exchange-interface     => ./exchange
- github.com/ipfs/go-unixfs                      => ./ipld/unixfs
- github.com/ipfs/go-merkledag                   => ./ipld/merkledag
- github.com/ipld/go-car                         => ./ipld/car

Fixes #215
Updates #202
  • Loading branch information
@Jorropo
Jorropo committed Mar 24, 2023
25 parents 664b3e5 + 00087bb + ae9364a + c2c98d0 + eeceee5 + b180aa1 + 2c8dec8 + 89bd5ed + b739af1 + a8baf45 + fbb6fad + 8aec54e + 7f24945 + 8169e7a + bc3a4d7 + 6199506 + 504dcf9 + a8700eb + cd0d3b1 + 62bb938 + 9f11814 + 54d20f0 + 18b8f6f + 790ce94 + c270c83 commit a4f8611
Show file tree
Hide file tree
Showing 492 changed files with 66,657 additions and 1,841 deletions.
11 changes: 6 additions & 5 deletions .github/workflows/gateway-sharness.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,24 +19,25 @@ jobs:
uses: actions/setup-go@v3
with:
go-version: 1.19.1
- name: Checkout go-libipfs
- name: Checkout boxo
uses: actions/checkout@v3
with:
path: go-libipfs
path: boxo
- name: Checkout Kubo
uses: actions/checkout@v3
with:
repository: ipfs/kubo
path: kubo
ref: 51570ebbb5dfa423e03d2d329de0252b924ad30d
- name: Install Missing Tools
run: sudo apt install -y socat net-tools fish libxml2-utils
- name: Restore Go Cache
uses: protocol/cache-go-action@v1
with:
name: ${{ github.job }}
- name: Replace go-libipfs in Kubo go.mod
- name: Replace boxo in Kubo go.mod
run: |
go mod edit -replace=github.com/ipfs/go-libipfs=../go-libipfs
go mod edit -replace=github.com/ipfs/boxo=../boxo
go mod tidy
cat go.mod
working-directory: kubo
Expand All @@ -47,5 +48,5 @@ jobs:
run: find . -maxdepth 1 -name "*gateway*.sh" -print0 | xargs -0 -I {} bash -c "echo {}; {}"
working-directory: kubo/test/sharness
- name: Run Kubo CLI Tests
run: go test -v -run=Gateway .
run: go test -run=Gateway .
working-directory: kubo/test/cli
6 changes: 3 additions & 3 deletions .github/workflows/go-test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
# Use -coverpkg=./..., so that we include cross-package coverage.
# If package ./A imports ./B, and ./A's tests also cover ./B,
# this means ./B's coverage will be significantly higher than 0%.
run: go test -v -shuffle=on -coverprofile=module-coverage.txt -coverpkg=./... ./...
run: go test -shuffle=on -coverprofile=module-coverage.txt -coverpkg=./... ./...
- name: Run tests (32 bit)
# can't run 32 bit tests on OSX.
if: matrix.os != 'macos' &&
Expand All @@ -58,14 +58,14 @@ jobs:
with:
run: |
export "PATH=$PATH_386:$PATH"
go test -v -shuffle=on ./...
go test -shuffle=on ./...
- name: Run tests with race detector
# speed things up. Windows and OSX VMs are slow
if: matrix.os == 'ubuntu' &&
contains(fromJSON(steps.config.outputs.json).skipOSes, matrix.os) == false
uses: protocol/[email protected]
with:
run: go test -v -race ./...
run: go test -race ./...
- name: Collect coverage files
shell: bash
run: echo "COVERAGES=$(find . -type f -name 'module-coverage.txt' | tr -s '\n' ',' | sed 's/,$//')" >> $GITHUB_ENV
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/test-examples.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
- name: Run tests
uses: protocol/[email protected]
with:
run: go test -v -shuffle=on ./...
run: go test -shuffle=on ./...
- name: Run tests (32 bit)
if: ${{ matrix.os != 'macos' }} # can't run 32 bit tests on OSX.
uses: protocol/[email protected]
Expand All @@ -46,9 +46,9 @@ jobs:
with:
run: |
export "PATH=${{ env.PATH_386 }}:$PATH"
go test -v -shuffle=on ./...
go test -shuffle=on ./...
- name: Run tests with race detector
if: ${{ matrix.os == 'ubuntu' }} # speed things up. Windows and OSX VMs are slow
uses: protocol/[email protected]
with:
run: go test -v -race ./...
run: go test -race ./...
54 changes: 27 additions & 27 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
<h1 align="center">
go-libipfs 🍌
Boxo 🍌
<br>
<img src="https://raw.githubusercontent.com/ipfs/go-libipfs/main/logo.svg" alt="go-libipfs logo" title="go-libipfs logo" width="200">
<img src="https://raw.githubusercontent.com/ipfs/boxo/main/logo.svg" alt="Boxo logo" title="Boxo logo" width="200">
<br>
</h1>
<p align="center" style="font-size: 1.2rem;">A library for building IPFS applications and implementations.</p>

<hr />

[![Go Test](https://github.com/ipfs/go-libipfs/actions/workflows/go-test.yml/badge.svg)](https://github.com/ipfs/go-libipfs/actions/workflows/go-test.yml)
[![Go Docs](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/ipfs/go-libipfs)
[![codecov](https://codecov.io/gh/ipfs/go-libipfs/branch/main/graph/badge.svg?token=9eG7d8fbCB)](https://codecov.io/gh/ipfs/go-libipfs)
[![Go Test](https://github.com/ipfs/boxo/actions/workflows/go-test.yml/badge.svg)](https://github.com/ipfs/boxo/actions/workflows/go-test.yml)
[![Go Docs](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/ipfs/boxo)
[![codecov](https://codecov.io/gh/ipfs/boxo/branch/main/graph/badge.svg?token=9eG7d8fbCB)](https://codecov.io/gh/ipfs/boxo)

<!-- TOC -->

- [About](#about)
- [Motivation](#motivation)
- [What kind of components does go-libipfs have?](#what-kind-of-components-does-go-libipfs-have)
- [Does go-libipfs == IPFS?](#does-go-libipfs--ipfs)
- [What kind of components does Boxo have?](#what-kind-of-components-does-boxo-have)
- [Does Boxo == IPFS?](#does-boxo--ipfs)
- [Is everything related to IPFS in the Go ecosystem in this repo?](#is-everything-related-to-ipfs-in-the-go-ecosystem-in-this-repo)
- [Getting started](#getting-started)
- [Should I add my IPFS component to go-libipfs?](#should-i-add-my-ipfs-component-to-go-libipfs)
- [Should I add my IPFS component to Boxo?](#should-i-add-my-ipfs-component-to-boxo)
- [Help](#help)
- [Governance and Access](#governance-and-access)
- [Release Process](#release-process)
Expand All @@ -31,75 +31,75 @@ go-libipfs 🍌

## About

go-libipfs is a component library for building IPFS applications and implementations in Go.
Boxo is a component library for building IPFS applications and implementations in Go.

Some scenarios in which you may find go-libipfs helpful:
Some scenarios in which you may find Boxo helpful:

* You are building an application that interacts with the IPFS network
* You are building an IPFS implementation
* You want to reuse some components of IPFS such as its Kademlia DHT, Bitswap, data encoding, etc.
* You want to experiment with IPFS

go-libipfs powers [Kubo](https://github.com/ipfs/kubo), which is [the most popular IPFS implementation](https://github.com/protocol/network-measurements/tree/master/reports),
Boxo powers [Kubo](https://github.com/ipfs/kubo), which is [the most popular IPFS implementation](https://github.com/protocol/network-measurements/tree/master/reports),
so its code has been battle-tested on the IPFS network for years, and is well-understood by the community.

### Motivation
**TL;DR** The goal of this repo is to help people build things. Previously users struggled to find existing useful code or to figure out how to use what they did find. We observed many running Kubo and using its HTTP RPC API. This repo aims to do better. We're taking the libraries that many were already effectively relying on in production and making them more easily discoverable and usable.

The maintainers primarily aim to help people trying to build with IPFS in Go that were previously either giving up or relying on the [Kubo HTTP RPC API](https://docs.ipfs.tech/reference/kubo/rpc/). Some of these people will end up being better served by IPFS tooling in other languages (e.g., Javascript, Rust, Java, Python), but for those who are either looking to write in Go or to leverage the set of IPFS tooling we already have in Go we’d like to make their lives easier.
The maintainers primarily aim to help people trying to build with IPFS in Go that were previously either giving up or relying on the [Kubo HTTP RPC API](https://docs.ipfs.tech/reference/kubo/rpc/). Some of these people will end up being better served by IPFS tooling in other languages (e.g., Javascript, Rust, Java, Python), but for those who are either looking to write in Go or to leverage the set of IPFS tooling we already have in Go we’d like to make their lives easier.

We’d also like to make life easier on ourselves as the maintainers by reducing the maintenance burden that comes from being the owners on [many repos](https://github.com/ipfs/kubo/issues/8543) and then use that time to contribute more to the community in the form of easier to use libraries, better implementations, improved protocols, new protocols, etc.
We’d also like to make life easier on ourselves as the maintainers by reducing the maintenance burden that comes from being the owners on [many repos](https://github.com/ipfs/kubo/issues/8543) and then use that time to contribute more to the community in the form of easier to use libraries, better implementations, improved protocols, new protocols, etc.

Go-libipfs is not exhaustive nor comprehensive--there are plenty of useful IPFS protocols, specs, libraries, etc. that are not in go-libipfs. The goal of go-libipfs is to provide cohesive and well-maintained components for common IPFS use cases.
Boxo is not exhaustive nor comprehensive--there are plenty of useful IPFS protocols, specs, libraries, etc. that are not in Boxo. The goal of Boxo is to provide cohesive and well-maintained components for common IPFS use cases.

## What kind of components does go-libipfs have?
## What kind of components does Boxo have?

Go-libipfs includes high-quality components useful for interacting with IPFS protocols, public and private IPFS networks, and content-addressed data, such as:
Boxo includes high-quality components useful for interacting with IPFS protocols, public and private IPFS networks, and content-addressed data, such as:

- Content routing (DHT, delegated content routing, providing)
- Data transfer (gateways, Bitswap, incremental verification)
- Naming and mutability (name resolution, IPNS)
- Interacting with public and private IPFS networks
- Working with content-addressed data

Go-libipfs aims to provide a cohesive interface into these components. Note that not all of the underlying components necessarily reside in this respository.
Boxo aims to provide a cohesive interface into these components. Note that not all of the underlying components necessarily reside in this respository.

### Does go-libipfs == IPFS?
### Does Boxo == IPFS?
No. This repo houses some IPFS functionality written in Go that has been useful in practice, and is maintained by a group that has long term commitments to the IPFS project

### Is everything related to IPFS in the Go ecosystem in this repo?

No. Not everything related to IPFS is intended to be in go-libipfs. View it as a starter toolbox (potentially among multiple). If you’d like to build an IPFS implementation with Go, here are some tools you might want that are maintained by a group that has long term commitments to the IPFS project. There are certainly repos that others maintainer that aren't included here (e.g., ipfs/go-car) which are still useful to IPFS implementations. It's expected and fine for new IPFS functionality to be developed that won't be part of go-libipfs.
No. Not everything related to IPFS is intended to be in Boxo. View it as a starter toolbox (potentially among multiple). If you’d like to build an IPFS implementation with Go, here are some tools you might want that are maintained by a group that has long term commitments to the IPFS project. There are certainly repos that others maintainer that aren't included here (e.g., ipfs/go-car) which are still useful to IPFS implementations. It's expected and fine for new IPFS functionality to be developed that won't be part of Boxo.

## Getting started
See [examples](./examples/README.md).

## Should I add my IPFS component to go-libipfs?
We happily accept external contributions! However, go-libipfs maintains a high quality bar, so code accepted into go-libipfs must meet some minimum maintenance criteria:
## Should I add my IPFS component to Boxo?
We happily accept external contributions! However, Boxo maintains a high quality bar, so code accepted into Boxo must meet some minimum maintenance criteria:

* Actively maintained
* Must be actively used by, or will be included in software that is actively used by, a significant number of users or production systems. Code that is not actively used cannot be properly maintained.
* Must have multiple engineers who are willing and able to maintain the relevant code in go-libipfs for a long period of time.
* If either of these changes, go-libipfs maintainers will consider removing the component from go-libipfs.
* Must have multiple engineers who are willing and able to maintain the relevant code in Boxo for a long period of time.
* If either of these changes, Boxo maintainers will consider removing the component from Boxo.
* Adequately tested
* At least with unit tests
* Ideally also including integration tests with other components
* Adequately documented
* Godocs at minimum
* Complex components should have their own doc.go or README.md describing the component, its use cases, tradeoffs, design rationale, etc.
* If the maintainers are not go-libipfs maintainers, then the component must include a CODEOWNERS file with at least two code owners who can commit to reviewing PRs
* If the maintainers are not Boxo maintainers, then the component must include a CODEOWNERS file with at least two code owners who can commit to reviewing PRs

If you have some experimental component that you think would benefit the IPFS community, we suggest you build the component in your own repository until it's clear that there's community demand for it, and then open an issue/PR in this repository to discuss including it in go-libipfs.
If you have some experimental component that you think would benefit the IPFS community, we suggest you build the component in your own repository until it's clear that there's community demand for it, and then open an issue/PR in this repository to discuss including it in Boxo.

## Help

If you have questions, feel free to open an issue. You can also find the go-libipfs maintainers in [Filecoin Slack](https://filecoin.io/slack/) at #go-libipfs-maintainers. (If you would like to engage via IPFS Discord or ipfs.io Matrix, please drop into the #ipfs-implementers channel/room or file an issue, and we'll get bridging from #go-libipfs-maintainers to these other chat platforms.)
If you have questions, feel free to open an issue. You can also find the Boxo maintainers in [Filecoin Slack](https://filecoin.io/slack/) at #Boxo-maintainers. (If you would like to engage via IPFS Discord or ipfs.io Matrix, please drop into the #ipfs-implementers channel/room or file an issue, and we'll get bridging from #Boxo-maintainers to these other chat platforms.)

## Governance and Access
See [CODEOWNERS](./docs/CODEOWNERS) for the current maintainers list. Governance for graduating additional maintainers hasn't been established. Repo permissions are all managed through [ipfs/github-mgmt](https://github.com/ipfs/github-mgmt).

## Release Process
To be documented: https://github.com/ipfs/go-libipfs/issues/170
To be documented: https://github.com/ipfs/boxo/issues/170

## Related Items
* [Initial proposal for "Consolidate IPFS Repositories" that spawned this project](https://github.com/ipfs/kubo/issues/8543)
Expand Down
4 changes: 2 additions & 2 deletions bitswap/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ wants those blocks.
```golang
import (
"context"
bitswap "github.com/ipfs/go-libipfs/bitswap"
bsnet "github.com/ipfs/go-libipfs/bitswap/network"
bitswap "github.com/ipfs/boxo/bitswap"
bsnet "github.com/ipfs/boxo/bitswap/network"
blockstore "github.com/ipfs/go-ipfs-blockstore"
"github.com/libp2p/go-libp2p-core/routing"
"github.com/libp2p/go-libp2p-core/host"
Expand Down
14 changes: 7 additions & 7 deletions bitswap/benchmarks_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,17 @@ import (
"testing"
"time"

"github.com/ipfs/go-libipfs/bitswap/internal/testutil"
blocks "github.com/ipfs/go-libipfs/blocks"
"github.com/ipfs/boxo/bitswap/internal/testutil"
blocks "github.com/ipfs/go-block-format"
protocol "github.com/libp2p/go-libp2p/core/protocol"

"github.com/ipfs/boxo/bitswap"
bsnet "github.com/ipfs/boxo/bitswap/network"
testinstance "github.com/ipfs/boxo/bitswap/testinstance"
tn "github.com/ipfs/boxo/bitswap/testnet"
mockrouting "github.com/ipfs/boxo/routing/mock"
cid "github.com/ipfs/go-cid"
delay "github.com/ipfs/go-ipfs-delay"
mockrouting "github.com/ipfs/go-ipfs-routing/mock"
"github.com/ipfs/go-libipfs/bitswap"
bsnet "github.com/ipfs/go-libipfs/bitswap/network"
testinstance "github.com/ipfs/go-libipfs/bitswap/testinstance"
tn "github.com/ipfs/go-libipfs/bitswap/testnet"
)

type fetchFunc func(b *testing.B, bs *bitswap.Bitswap, ks []cid.Cid)
Expand Down
18 changes: 9 additions & 9 deletions bitswap/bitswap.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@ import (
"context"
"fmt"

"github.com/ipfs/go-libipfs/bitswap/client"
"github.com/ipfs/go-libipfs/bitswap/internal/defaults"
"github.com/ipfs/go-libipfs/bitswap/message"
"github.com/ipfs/go-libipfs/bitswap/network"
"github.com/ipfs/go-libipfs/bitswap/server"
"github.com/ipfs/go-libipfs/bitswap/tracer"
"github.com/ipfs/boxo/bitswap/client"
"github.com/ipfs/boxo/bitswap/internal/defaults"
"github.com/ipfs/boxo/bitswap/message"
"github.com/ipfs/boxo/bitswap/network"
"github.com/ipfs/boxo/bitswap/server"
"github.com/ipfs/boxo/bitswap/tracer"
"github.com/ipfs/go-metrics-interface"

blockstore "github.com/ipfs/boxo/blockstore"
exchange "github.com/ipfs/boxo/exchange"
blocks "github.com/ipfs/go-block-format"
"github.com/ipfs/go-cid"
blockstore "github.com/ipfs/go-ipfs-blockstore"
exchange "github.com/ipfs/go-ipfs-exchange-interface"
blocks "github.com/ipfs/go-libipfs/blocks"
logging "github.com/ipfs/go-log"
"github.com/libp2p/go-libp2p/core/peer"

Expand Down
16 changes: 8 additions & 8 deletions bitswap/bitswap_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,19 @@ import (
"testing"
"time"

"github.com/ipfs/boxo/bitswap"
bsmsg "github.com/ipfs/boxo/bitswap/message"
"github.com/ipfs/boxo/bitswap/server"
testinstance "github.com/ipfs/boxo/bitswap/testinstance"
tn "github.com/ipfs/boxo/bitswap/testnet"
"github.com/ipfs/boxo/internal/test"
mockrouting "github.com/ipfs/boxo/routing/mock"
blocks "github.com/ipfs/go-block-format"
cid "github.com/ipfs/go-cid"
detectrace "github.com/ipfs/go-detect-race"
blocksutil "github.com/ipfs/go-ipfs-blocksutil"
delay "github.com/ipfs/go-ipfs-delay"
mockrouting "github.com/ipfs/go-ipfs-routing/mock"
ipld "github.com/ipfs/go-ipld-format"
"github.com/ipfs/go-libipfs/bitswap"
bsmsg "github.com/ipfs/go-libipfs/bitswap/message"
"github.com/ipfs/go-libipfs/bitswap/server"
testinstance "github.com/ipfs/go-libipfs/bitswap/testinstance"
tn "github.com/ipfs/go-libipfs/bitswap/testnet"
blocks "github.com/ipfs/go-libipfs/blocks"
"github.com/ipfs/go-libipfs/internal/test"
tu "github.com/libp2p/go-libp2p-testing/etc"
p2ptestutil "github.com/libp2p/go-libp2p-testing/netutil"
peer "github.com/libp2p/go-libp2p/core/peer"
Expand Down
14 changes: 7 additions & 7 deletions bitswap/client/bitswap_with_sessions_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,16 @@ import (
"testing"
"time"

"github.com/ipfs/boxo/bitswap"
"github.com/ipfs/boxo/bitswap/client/internal/session"
testinstance "github.com/ipfs/boxo/bitswap/testinstance"
tn "github.com/ipfs/boxo/bitswap/testnet"
"github.com/ipfs/boxo/internal/test"
mockrouting "github.com/ipfs/boxo/routing/mock"
blocks "github.com/ipfs/go-block-format"
cid "github.com/ipfs/go-cid"
blocksutil "github.com/ipfs/go-ipfs-blocksutil"
delay "github.com/ipfs/go-ipfs-delay"
mockrouting "github.com/ipfs/go-ipfs-routing/mock"
"github.com/ipfs/go-libipfs/bitswap"
"github.com/ipfs/go-libipfs/bitswap/client/internal/session"
testinstance "github.com/ipfs/go-libipfs/bitswap/testinstance"
tn "github.com/ipfs/go-libipfs/bitswap/testnet"
blocks "github.com/ipfs/go-libipfs/blocks"
"github.com/ipfs/go-libipfs/internal/test"
tu "github.com/libp2p/go-libp2p-testing/etc"
)

Expand Down
Loading

0 comments on commit a4f8611

Please sign in to comment.