Documentation

Getting Started with Crucible

Get started using Crucible. Feel free to make an account, create projects, and upload data. Data files are stored in Google Cloud Storage under the Molecular Foundry account.

Access to data through the Crucible project is limited to platform administrators, members of the Crucible project associated with the dataset, and the instrument service account for the instrument associated with the dataset. A data retention policy for raw data files is under discussion; in the meantime, we recommend keeping your own copies and using Crucible for backup, access, and organization.

If you have questions or would like to use Crucible more extensively, please reach out to the team or submit a User Proposal to the Molecular Foundry.

Make an account

Go to the Crucible Explorer and sign in to create an account.

Install the Python client and CLI

nano-crucible provides both the crucible command-line tool and the Python client.

pip install nano-crucible

PyPI · GitHub · Documentation

Configure the client

This stores your API URL and API key so the CLI and Python client can authenticate as you.

crucible config init

Create a project

crucible project create --project-id <your-project-name>

A project name may not contain spaces or punctuation other than hyphens. Run the command with no flags to be prompted for each field, or supply them up front:

crucible project create --project-id <your-project-name> -o "LBNL" -e "lead@lbl.gov" \
    --title "Silicon Wafer Study"

Add users to the project

Add everyone you are working with. A user can be named by email, username, ORCID, or MFID.

crucible project add-user <your-project-name> --user <user email>
crucible project add-user <your-project-name> --user <username> --role editor

Each member should now see the project at the Crucible Explorer.

Add an instrument

Command line

crucible instrument create

crucible instrument create -n "titanx" --instrument-id titanx --location "B72-201"

Python

instrument_id, instrument_name, and location are required; the rest are optional.

from crucible import CrucibleClient
from crucible.models import Instrument

client = CrucibleClient()  # reads the config written by `crucible config init`

client.instruments.create(
    Instrument(
        instrument_id="titanx",
        instrument_name="titanx",
        location="B72-201",
        manufacturer="FEI",
        model="Titan 80-300",
        instrument_type="TEM",
    )
)

Create datasets

There are four ways to create a dataset. They all produce the same kind of record, so use whichever fits your workflow.

From the web explorer

Go to crucible.lbl.gov/explore.
Navigate to your project.
Choose New Dataset in the left-hand side panel.
Fill in the information.
Click Create.

From the command line

# Create a dataset record without files
crucible dataset create --project-id <your-project-name> --name "Planned experiment"

# Generic upload (server assigns mfid)
crucible dataset create -i file1.dat file2.csv --project-id <your-project-name>

# Upload with metadata and keywords
crucible dataset create -i data.csv --project-id <your-project-name> \
    --metadata '{"temperature": 300, "pressure": 1.0}' \
    --keywords "experiment,thermal" -m "thermal_analysis"

From Python

from crucible import CrucibleClient
from crucible.models import Dataset

client = CrucibleClient()

dataset = Dataset(
    dataset_name="0105 - 1429 Diffraction 70000 x HAADF 1.44 µm",
    project_id="<your-project-name>",
    instrument_id="titanx",
    measurement="STEM Diffraction EDS",
)

client.datasets.create(
    dataset,
    files=["0105 - 1429 Diffraction 70000 x HAADF 1.44 µm.emd"],
    scientific_metadata={"magnification": 70000, "field_of_view_um": 1.44},
    keywords=["diffraction", "HAADF"],
)

datasets.create() takes the dataset first, then keyword arguments:

Argument	Default	Purpose
`scientific_metadata`	`None`	Dict of metadata attached to the dataset
`keywords`	`None`	List of keyword strings
`files`	`None`	Local paths to upload, or `AssociatedFile` records for data that lives elsewhere
`upload_files`	`True`	Set `False` to catalog local paths in place instead of uploading
`ingestor`	`None`	Named ingestor used to parse the files
`verbose`	`False`	Print upload progress

From a local upload UI

For instrument computers, the upload UIs give you a drag-and-drop front end. Instructions for running the app are in the repository README.

git clone https://github.com/MolecularFoundryCrucible/crucible-upload-uis.git

You will need a service account

The app runs with a service account API key and permissions. Request one on the Discord or by emailing mkwall@lbl.gov, then bind it to your instrument:

crucible instrument bind-sa INSTRUMENT_MFID SERVICE_ACCOUNT_ID

Create samples

There are three ways to create a sample. They all produce the same kind of record, so use whichever fits your workflow.

From the web explorer

Go to crucible.lbl.gov/explore.
Navigate to your project.
Choose New Sample in the left-hand side panel.
Fill in the information.
Click Create.

From the command line

crucible sample create

crucible sample create -n "Silicon Wafer A" --project-id <your-project-name> \
    --description "Test sample" --type substrate

From Python

from crucible.models import Sample

client.samples.create(
    Sample(
        sample_name="Silicon Wafer A",
        project_id="<your-project-name>",
        sample_type="substrate",
        description="Test sample",
    ),
    scientific_metadata={"thickness_um": 525},
)

Create relationships

Records are linked by MFID. Samples and datasets share the same two relationship types: is_derived_from and is_part_of.

# sample → sample
client.samples.link(parent_mfid, child_mfid, relationship_type="is_part_of")
client.samples.link(parent_mfid, child_mfid, relationship_type="is_derived_from")

# dataset → dataset
client.datasets.link(parent_mfid, child_mfid, relationship_type="is_part_of")
client.datasets.link(parent_mfid, child_mfid, relationship_type="is_derived_from")

# dataset → sample
client.datasets.link_sample(dataset_mfid, sample_mfid)

Get additional support

Submit a User Proposal Join the Discord

On this page