Upload a folder

A folder upload sends every file in a folder (or a .zip file) and gives you one report. Ophiolite reads the start of each file to find out what it is, adds every file it can read as a private file of the project, and says for every other file why it was not added. Nothing is guessed: a file whose values need explaining (a grid’s coordinate system, whether a horizon is time or depth) waits for you to say what they mean.

A folder upload holds at most 500 files. Each file keeps the limits of its kind, listed on File kinds.

In the Workspace

1. Open Connect → Add files (the one place to add data; see Add data). Press Choose a folder or Choose files (a .zip file is opened as a folder), or drop a folder on “Drop files or a folder here”. A browser that cannot choose folders says “This browser cannot choose a folder. Choose files, or put the folder in a .zip file.”
2. Each file is read first. The read report lists every file (File, What it is, What was read) and asks each kind’s open questions once, every value preselected Not stated. Fill in Source attribution (required), Who may be given access and Well context notes once for the whole folder, tick that you have permission to keep these files, make results from them and share them within this audience, and press Add 23 files.
3. “Adding files… 7 of 23. Keep this tab open until it finishes.” Stop ends the upload; what was added stays. If the tab is closed or the network is lost, the Connect page says “Your upload of Tubbergen-Mander-2 stopped after 14 of 23 files. Choose the same folder again to continue; files already added are not sent twice.”
4. The report opens: “Tubbergen-Mander-2 - 23 files - Finished with files not added”, with one row per file (File, What it is, What was read, Result). Download report (CSV) and (JSON) save it; Share report lets named project members read the report (not the files). Every upload you made is listed under Your uploads.

Result	What it means
Added	A new private file of the project. “What was read” says, for a well log, its rows, curves and missing samples.
Already here	“Same content and settings as TUM-02/…, added earlier in this folder.” Nothing new was stored.
Not read	The file is of a kind Ophiolite reads, but this copy cannot be read: “The file is damaged or is not plain text (UTF-8). Nothing was repaired; export a UTF-8 LAS 2.0 copy.”
Not supported	“No reader recognises this file.” The sentence lists the kinds that are read.
Needs your decision	The file waits for you; the row says why and offers the choices below.
Cancelled	You stopped the upload or skipped the file.

Decisions. The row of a file that needs a decision finishes in the report:

• “A file with this path is already in the project with different content.” Add as a separate file or Skip.
• “This file is already in the project with other settings.” Add as a new file or Skip.
• Files whose values need explaining (grids, points, seismic, tops, deviation, faults) are asked about in the read report before anything is sent, so they are added with your answers. A file added from Python or the command line without its answers waits, grouped by kind: “Say what the values mean”. Fill in the form once for the ticked files; a field you leave alone is “Not stated”. Press Add n files, Add with everything “Not stated” or Skip. Each file shows the values it was added with.
• Files that need an import recipe: see below.

Wellbores. A well log names its well in its header. When one wellbore of the project has that name, the report proposes the link: “16 files can be linked to a wellbore from the well name in their header. Nothing is linked until you accept.” Press Link these 16 files, or Link / Not this one per file. A file whose header names no wellbore of the project keeps the ordinary Belongs to wellbore choice.

In Python

from ophiolite import Client

client = Client.from_configuration("configuration.json")
folder = "Tubbergen-Mander-2"         # a folder, or a .zip file
grids = "Grids"

report = client.upload_runs.upload(folder, attribution="Source: NLOG.NL", audience=[], rights_confirmed=True)
print(report.label, report.counts)    # Finished with files not added {'added': 20, 'already-here': 1, ...}
for row in report.rows():
    if row["result"] not in ("Added", "Already here"):
        print(row["path"], row["result"], row["reason"])
report.save("report.csv")             # path, result, kind, read, reason, asset; a .json path saves the whole report

Calling upload again for the same folder after an interruption continues the unfinished upload and sends only the files not yet added; new=True starts another. The report of a finished upload can be read again with client.upload_runs.status(report.run_id).

Link the files whose header names a wellbore of the project:

links = [(item["ordinal"], item["proposal"]["entity_id"]) for item in report.items if item.get("proposal")]
report, linked = client.upload_runs.associate(report.run_id, links)
print(linked, "files linked")

Say what the values of one kind mean (other kinds are not touched), or skip what still needs a decision:

report = client.upload_runs.upload(grids, attribution="Source: NLOG.NL", audience=[], rights_confirmed=True,
                                   declare={"esri-ascii-grid/1": {"crs": "EPSG:28992", "z_meaning": "depth"}})
print(report.counts["added"], [item["declared"] for item in report.items])

skip_decisions=True skips every file that still needs a decision, and associate_matches=True links each file to the one wellbore its header names in the same call.

From the command line

ophiolite upload ~/data/Tubbergen-Mander-2 --attribution "Source: NLOG.NL" --audience alice \
  --rights-confirmed --associate-matches --report report.csv

--declare esri-ascii-grid/1:crs=EPSG:28992 answers the files of one kind, --skip-decisions skips the rest, and --new starts another upload of a folder you uploaded before. The command exits 0 when every file was added or is already here, 4 when any was not (the report is still written), and 1 when it was refused before anything was sent.

Files that need an import recipe

Points from Petrel and horizons from OpendTect are read by an import recipe: a saved description of the file’s columns and what they mean. Recipes are set up on their own screen, which comes with a later release. A folder upload reads these files with the standard layout; a file the standard layout cannot read waits as “Needs your decision” with Show the Python steps and Skip. Until the recipe screen arrives, add such a file from Python, saying what its values mean:

horizon = client.upload_data("horizon_xyz.txt", profile="opendtect-horizon-ascii/1", name="Top reservoir (time)",
                             declared={"crs": "EPSG:23031", "z_meaning": "time", "z_unit": "s"},
                             attribution="Source: NLOG.NL", audience=[], rights_confirmed=True)
print(horizon.asset_id)

Then skip the row in the report; the file you added from Python is the one that stays.

When a folder is refused

You see	Why
“A folder upload holds at most 500 files; this folder has 612. Choose a smaller folder.”	more than 500 files; nothing was sent
“This zip file cannot be opened. Nothing was sent.”	the .zip file is damaged
“This zip file contains ../x.las, which is not allowed. Nothing was sent.”	an entry outside the folder, a link, an entry over the file limit, or more than 100 times its packed size or 512 MiB unpacked
“Adding data requires permission to edit this project. Ask a project administrator.”	you may read this project but not add to it

A file refused for the deployment’s capacity says so in its own row (“Upload capacity reached: …”); the other files are still tried.

The notebook A well folder in one report does the same with a small folder.