Filecoin

You can publish Old Faithful-based archives with a Filecoin storage provider. This would allow you to use the Filecoin retrieval modes with your preferred provider.

For warehouse operators

Flow v1

The basic flow of uploading Old Faithful data to Filecoin involves:

  1. Generating the CAR file for the epoch (or cloning it)

  2. Splitting the car file into suitable sized chunks for ingest into filecoin (32 GiB), saving the metadata.yaml produced as part of the process (will be needed for retrievals).

  3. Using boost to create deals for the split files, creating a deals.csv file in the process (see format below).

  4. Once the deals are ingested, you can use the Filecoin v1 version of retrievals.

Deals.csv file format

The deals.csv file format should contain the following headers as the first line (and then the corresponding values under each value):

  • provider

  • deal_uuid

  • file_name

  • URL

  • commp_piece_cid

  • file_size

  • padded_size

  • payload_cid

Sample deals.csv
provider,deal_uuid,file_name,url,commp_piece_cid,file_size,padded_size,payload_cid
f02366527,db161970-b4ab-4a05-bae4-b0e559a48806,sp-epoch-100-baga6ea4seaqiogvxkc4cjgbimal52zgaqwo5ugcvy2bc72nrerro3wtzy47jepi.car,https://filecoin-car-storage-cdn.b-cdn.net/100/sp-epoch-100-baga6ea4seaqiogvxkc4cjgbimal52zgaqwo5ugcvy2bc72nrerro3wtzy47jepi.car,baga6ea4seaqiogvxkc4cjgbimal52zgaqwo5ugcvy2bc72nrerro3wtzy47jepi,30000000796,34359738368,bafyreibqt2nvroysxlxctgb52xxn27ectsllv2xyka4qar7ga6vupmbs3i
f02366527,cd27c9bd-4d26-4deb-9579-6821e42bdbdd,sp-epoch-100-baga6ea4seaqowdba7qkhqxw3umullabn7mx377we7vu3klguccuderfqclzpsiy.car,https://filecoin-car-storage-cdn.b-cdn.net/100/sp-epoch-100-baga6ea4seaqowdba7qkhqxw3umullabn7mx377we7vu3klguccuderfqclzpsiy.car,baga6ea4seaqowdba7qkhqxw3umullabn7mx377we7vu3klguccuderfqclzpsiy,30000000081,34359738368,bafyreibqt2nvroysxlxctgb52xxn27ectsllv2xyka4qar7ga6vupmbs3i
f02366527,c9237f39-0cfc-48ba-a4b8-7163f4543305,sp-epoch-100-baga6ea4seaqhky3ydiwp5toduhxpzt7oxm5cctuvrjbmkq5gzf3s6arayindoii.car,https://filecoin-car-storage-cdn.b-cdn.net/100/sp-epoch-100-baga6ea4seaqhky3ydiwp5toduhxpzt7oxm5cctuvrjbmkq5gzf3s6arayindoii.car,baga6ea4seaqhky3ydiwp5toduhxpzt7oxm5cctuvrjbmkq5gzf3s6arayindoii,2937660613,4294967296,bafyreibqt2nvroysxlxctgb52xxn27ectsllv2xyka4qar7ga6vupmbs3i
f02366527,473a2c70-8f0f-498b-b539-d3f0e50a4579,sp-epoch-100-baga6ea4seaqiogvxkc4cjgbimal52zgaqwo5ugcvy2bc72nrerro3wtzy47jepi.car,https://filecoin-car-storage-cdn.b-cdn.net/100/sp-epoch-100-baga6ea4seaqiogvxkc4cjgbimal52zgaqwo5ugcvy2bc72nrerro3wtzy47jepi.car,baga6ea4seaqiogvxkc4cjgbimal52zgaqwo5ugcvy2bc72nrerro3wtzy47jepi,30000000796,34359738368,bafyreibqt2nvroysxlxctgb52xxn27ectsllv2xyka4qar7ga6vupmbs3i
f02366527,6b0a5754-371c-4cd9-8553-c26dc37a0763,sp-epoch-100-baga6ea4seaqowdba7qkhqxw3umullabn7mx377we7vu3klguccuderfqclzpsiy.car,https://filecoin-car-storage-cdn.b-cdn.net/100/sp-epoch-100-baga6ea4seaqowdba7qkhqxw3umullabn7mx377we7vu3klguccuderfqclzpsiy.car,baga6ea4seaqowdba7qkhqxw3umullabn7mx377we7vu3klguccuderfqclzpsiy,30000000081,34359738368,bafyreibqt2nvroysxlxctgb52xxn27ectsllv2xyka4qar7ga6vupmbs3i
f02366527,63e14784-ac0b-44a9-8e3b-0395b654a519,sp-epoch-100-baga6ea4seaqhky3ydiwp5toduhxpzt7oxm5cctuvrjbmkq5gzf3s6arayindoii.car,https://filecoin-car-storage-cdn.b-cdn.net/100/sp-epoch-100-baga6ea4seaqhky3ydiwp5toduhxpzt7oxm5cctuvrjbmkq5gzf3s6arayindoii.car,baga6ea4seaqhky3ydiwp5toduhxpzt7oxm5cctuvrjbmkq5gzf3s6arayindoii,2937660613,4294967296,bafyreibqt2nvroysxlxctgb52xxn27ectsllv2xyka4qar7ga6vupmbs3i

Flow v2

Flow v2 for generating CAR files will be necessary to support Filecoin v2 retrievals. It is still in progress and involves the use of a new split tool.

For storage providers

If you are a Filecoin storage provider, you can participate in the Old Faithful project by partnering with a Solana warehouse node operator that generates CAR file deals.

PreviousMaking a local copyNextValidation

Last updated