Read part of a volume

A window is part of a seismic volume or a 2D line at an exact revision: some inlines, some crosslines and some samples, or some traces and samples of a line. You ask for the window’s document, then for its pieces one by one. Nothing else is downloaded: the original file stays where it is. Every example on this page runs in the test suite against a deployment, exactly as written.

The examples read these variables:

Variable	Holds
OPHIOLITE_URL	the deployment’s address, for example https://ophiolite.example.org
OPHIOLITE_PROJECT	the project id
OPHIOLITE_ACCESS_KEY	a project access key or a service account’s key (oph_key_...), from your account page
ASSET, REVISION	the volume or line and its exact revision
WINDOW	the window, for example inline=102:104&crossline=305:315&sample=3:5&by=label; empty for the whole volume

What a window is

Each range is first:last, both ends included, or one number. Inlines and crosslines are given by their numbers from the trace headers (by=label) or by position from 0 (by=ordinal); samples are always positions from 0. A 2D line has trace (position in file order) and sample. The rules are those of the contract ophiolite.volume-window/1, which every deployment serves at /api/v1/contracts/assets/v1/volume-window-schema.json. The window routes are supported operations: they change only as the versioning policy allows.

curl -sS --fail-with-body -H "Authorization: Bearer $OPHIOLITE_ACCESS_KEY" \
  "$OPHIOLITE_URL/api/v1/projects/$OPHIOLITE_PROJECT/scientific-assets/$ASSET/revisions/$REVISION/volume-windows?$WINDOW"

The answer is the window’s document: its window_id, the axes with the label of every trace, the shape, the stored sample format, the bytes it holds and how it is cut into pieces. It holds no sample.

With curl

The block asks for the document and every piece. It checks each piece’s body against X-Content-SHA256 and its X-Ophiolite-Window, X-Ophiolite-Revision and X-Ophiolite-Piece headers against the document, then that every piece arrived once and the bytes add up. It prints the window’s sha256, which you compute: the deployment never states one. It needs curl, jq and sha256sum.

base="$OPHIOLITE_URL/api/v1/projects/$OPHIOLITE_PROJECT/scientific-assets/$ASSET/revisions/$REVISION/volume-windows"
auth="Authorization: Bearer $OPHIOLITE_ACCESS_KEY"
curl -sS --fail-with-body -H "$auth" "$base?$WINDOW" > window.json
window=$(jq -r .window_id window.json); count=$(jq -r .pieces.count window.json)
[ "$(jq -r .asset_id window.json) $(jq -r .revision window.json)" = "$ASSET $REVISION" ] || { echo "Another asset or revision" >&2; exit 1; }
header() { grep -i "^$1:" piece.headers | cut -d' ' -f2- | tr -d '\r'; }
: > window.bin
for ((i = 0; i < count; i++)); do
  curl -sS --fail-with-body -H "$auth" -D piece.headers -o piece.bin "$base/pieces/$i?${WINDOW:+$WINDOW&}window=$window"
  [ "$(header X-Ophiolite-Window)" = "$window" ] || { echo "Piece $i belongs to another window" >&2; exit 1; }
  [ "$(header X-Ophiolite-Revision)" = "$REVISION" ] || { echo "Piece $i is of another revision" >&2; exit 1; }
  [ "$(header X-Ophiolite-Piece)" = "$i/$count" ] || { echo "Piece $i is missing or repeated" >&2; exit 1; }
  [ "$(sha256sum piece.bin | cut -d' ' -f1)" = "$(header X-Content-SHA256)" ] || { echo "Piece $i is damaged" >&2; exit 1; }
  cat piece.bin >> window.bin
done
[ "$(stat -c %s window.bin)" = "$(jq -r .bytes window.json)" ] || { echo "The window is not complete" >&2; exit 1; }
sha256sum window.bin | cut -d' ' -f1

window.bin then holds the stored 4-byte big-endian words, trace by trace in the order of the axes.

With Python, no Ophiolite package

The same read with the standard library only (urllib.request, json, hashlib, struct). It makes every check the document lists under verification.client: identity, axes, shape, piece identity, piece digest and completeness. It decodes IBM (format 1) and IEEE (format 5) words and prints the values and the window’s sha256 as JSON.

import hashlib, json, math, os, struct, urllib.request

asset, revision, query = os.environ["ASSET"], os.environ["REVISION"], os.environ.get("WINDOW", "")
base = "%s/api/v1/projects/%s/scientific-assets/%s/revisions/%s/volume-windows" % (
    os.environ["OPHIOLITE_URL"], os.environ["OPHIOLITE_PROJECT"], asset, revision)
key = {"Authorization": "Bearer " + os.environ["OPHIOLITE_ACCESS_KEY"]}


def get(url):
    with urllib.request.urlopen(urllib.request.Request(url, headers=key)) as answer:
        return answer.headers, answer.read()


def refuse(why):
    raise SystemExit("Refused: " + why)


doc = json.loads(get(base + "?" + query)[1])
if (doc["asset_id"], doc["revision"]) != (asset, revision): refuse("another asset or revision")
if [a["name"] for a in doc["axes"]] not in (["inline", "crossline", "sample"], ["trace", "sample"]): refuse("not the axes of a volume or a line")
if any(a["count"] != a["last"] - a["first"] + 1 or len(a.get("labels", range(a["count"]))) != a["count"] for a in doc["axes"]): refuse("an axis does not match its range")
if doc["shape"] != [a["count"] for a in doc["axes"]] or doc["bytes"] != math.prod(doc["shape"]) * 4: refuse("the shape does not match the axes")

body, seen, count = b"", [], doc["pieces"]["count"]
for i in range(count):
    headers, piece = get("%s/pieces/%d?%s" % (base, i, "&".join(filter(None, [query, "window=" + doc["window_id"]]))))
    if headers["X-Ophiolite-Window"] != doc["window_id"]: refuse("piece %d belongs to another window" % i)
    if headers["X-Ophiolite-Revision"] != revision: refuse("piece %d is of another revision" % i)
    if hashlib.sha256(piece).hexdigest() != headers["X-Content-SHA256"]: refuse("piece %d is damaged" % i)
    seen.append(headers["X-Ophiolite-Piece"])
    body += piece
if seen != ["%d/%d" % (i, count) for i in range(count)]: refuse("a piece is missing or repeated")
if len(body) != doc["bytes"]: refuse("the window is not complete")


def ibm(word):  # IBM System/360: sign, base-16 exponent biased by 64, 24-bit fraction
    value = math.ldexp(word & 0xFFFFFF, 4 * (((word >> 24) & 0x7F) - 64) - 24)
    return -value if word >> 31 else value


n = len(body) // 4
values = list(struct.unpack(">%df" % n, body)) if doc["samples"]["format_code"] == 5 else [ibm(w) for w in struct.unpack(">%dI" % n, body)]
shape = doc["shape"]
traces = [values[t * shape[-1]:(t + 1) * shape[-1]] for t in range(n // shape[-1])]
if len(shape) == 3: traces = [traces[i * shape[1]:(i + 1) * shape[1]] for i in range(shape[0])]
print(json.dumps({"window_id": doc["window_id"], "shape": shape, "values": traces, "sha256": hashlib.sha256(body).hexdigest()}))

With the Python SDK

client is your client on a key: Client(url, project, Credential.bearer(os.environ["OPHIOLITE_ACCESS_KEY"])) (see Build on Ophiolite). read_window reads and checks a window and holds it decoded; iter_window gives one checked piece at a time, so a whole volume never sits in memory.

window = client.read_window(asset, revision, inline="1:2", crossline="1:3", sample="3:5", by="ordinal")
print(window.shape, window.window_digest)
values = window.values                          # [inline][crossline][sample], as stored

reader = client.iter_window(asset, revision)    # the whole volume, one piece at a time
for piece in reader:
    print("piece", piece.index + 1, "of", piece.count, "holds", len(piece.values), "traces")
print(reader.complete, reader.window_digest)

A window the deployment or the SDK refuses raises WindowRefused; its reason is one of the codes below.

To a SEG-Y file

In a folder with the project’s configuration.json, ophiolite get --window writes the window as window.sgy, with window.json (the document and the window’s sha256) and descriptor.json beside it. The trace headers are the original’s; only the sample count and the start time change.

ophiolite get --item "$ASSET" --output window --window inline=102:104,crossline=305:315,sample=2:4,by=label

Any SEG-Y reader opens it, for example segyio:

import segyio

with segyio.open("window/window.sgy", iline=189, xline=193) as f:
    inlines, crosslines, times = list(f.ilines), list(f.xlines), list(f.samples)
    cube = segyio.tools.cube(f)                 # [inline][crossline][sample]

Long reads

A read of a large volume is many requests, one per piece, and each is checked against the credential as it is asked. Use a project access key or a service account’s key for it, not a browser sign-in: a Workspace or notebook sign-in hands scripts a delegate that lasts five minutes. A read on a key goes on as long as the key is valid.

import json

reader = client.iter_window(asset, revision)
with open("volume.json", "w") as out:
    for piece in reader:
        json.dump({"piece": piece.index, "traces": piece.traces, "values": piece.values}, out)
        out.write("\n")
print(reader.complete, reader.window_digest)

For a job that must keep running when people move on (a nightly read), use a service account’s key in the job’s own environment:

import os
from ophiolite import Client, Credential

job = Client(os.environ["OPHIOLITE_URL"], os.environ["OPHIOLITE_PROJECT"], Credential.bearer(os.environ["OPHIOLITE_ACCESS_KEY"]))
volume = job.read_window(asset, revision)
print(volume.shape, volume.window_digest)

The read stops at the next piece, and nothing more of the volume is read, when:

What you see	Why	What to do
Project access key revoked or expired	the key was removed, reached its end date, or its service account was disabled	Make a new key (or enable the account) and read the window again.
Owner no longer has this access	the key’s owner left the project or lost read access	Ask a project administrator for access, or use a service account’s key.
Project access key scope denied	the key belongs to another project	Use a key made for this project.
Asset unavailable	the volume is no longer shared with you	Ask its author to share it again.
Automation scope denied	a browser sign-in’s five-minute delegate ended	Read with a key instead.

Refusals

A refused window says what to change, and nothing was read. The SDK raises WindowRefused with the code; ophiolite get exits with code 1.

What you see	Code	What to do
The inline range ’…’ is not a number or two numbers joined by a colon …	malformed	Give first:last or one number; give each parameter once, and by as label or ordinal.
This volume takes no parameter ’…’: ask with …	unknown-parameter	Use only the parameters the sentence names.
Say whether inline and crossline are labels or ordinals: add by=label or by=ordinal.	selection-required	Add by=label or by=ordinal.
The … range … runs backwards: give the first and then the last, both included.	reversed	Give the smaller number first.
… is outside this volume: …	outside	Stay within the ranges the sentence gives.
… is not on this volume’s grid: …	off-grid	Use a line number on the grid’s step.
A 2D line is addressed by trace position in file order, not by trace number: …	line-by-label	Give trace positions from 0.
This asset is not a seismic volume or a 2D seismic line, so it has no part to read.	not-a-volume	Choose a volume or a line.
Reading part of a volume is not available on this deployment yet. …	not-available-yet	Read a slice or the whole original until the deployment is upgraded.
This window no longer matches its window_id: ask for the window again and use its new window_id.	window-changed	Ask for the document again.
This window starts at … ms; this file format …	inexact-crop	Start the window at sample 0 or at a whole millisecond.
This volume’s file states SEG-Y revision …; a window file is written from revision 0 or 1 files only.	revision-unsupported	Read the window in Python instead.
This volume keeps its … where a window file states its start time or sample count, …	header-positions-overlap	Read the window in Python instead.
A 2D line has no window file; get writes its original.	window-volume-only	Get the line without --window, or read its window in Python.
Give a window, for example —window inline=100:104,by=label, or call Client.download_original for the whole file.	volume-needs-window	Add --window, or download the original.
A window is written as a file: give a folder to write it into.	window-needs-output	Add --output.