Particle accelerators · a primer

Machines that turn fields into controlled beams.

Simplified accelerator beamline

From particle source to experiment

Particle bunchRF accelerationBeam magnetsLive telemetry
How an accelerator beamline works A compact bunch of three equal-size smiling particles leaves a source, accelerates, shifts from orange to red, and compresses more tightly together as radio-frequency cavities add energy. The bunch bends through a dipole magnet, narrows through quadrupole magnets, passes through beam diagnostics, and reaches an experiment. Animated signals from RF probes, magnet-current monitors, beam-position monitors, and beam-loss monitors travel to the control system.
Follow one compact three-particle bunch from source to experiment: the equal-size particles accelerate, deepen from orange to red, and compress even closer together; magnets bend and focus them while diagnostics measure the beam.

Accelerators at scale

Kilometers of precision infrastructure.

Particle accelerators use electric fields to increase the energy of charged particles and magnetic fields to bend, steer, and focus them into controlled beams. Flagship research accelerators are billion-dollar scientific facilities whose thousands of components must operate together with extraordinary precision.

At Los Alamos, the backbone of LANSCE is an 800 MeV linear accelerator approximately one kilometer long; at CERN, the Large Hadron Collider guides protons and ions around a 27-kilometer ring. Continuous diagnostics and control systems keep these enormous machines stable, safe, and useful for experiments.

Applications

Why build particle accelerators?

01

Fundamental discovery

Collide or scatter particles to probe matter, forces, and the early universe.

02

Medicine

Treat cancer with particle beams and produce radioisotopes for imaging and therapy.

03

Materials & chemistry

Use intense x-rays and neutrons to resolve structures and follow fast processes.

04

Industry & security

Inspect components, modify materials, sterilize products, and image concealed objects.

Facilities represented in this study

Different machines, related operational data.

  • LANSCE≈1 km proton linear accelerator · Los Alamos
  • LHC27 km proton and ion collider · CERN
  • SNSPulsed proton accelerator and neutron source · Oak Ridge
  • LCLSElectron linac and x-ray free-electron laser · SLAC
  • CEBAFRecirculating electron accelerator · Jefferson Lab

Research at a glance

From specialist models to one reusable accelerator representation.

01

Motivation

Prior accelerator ML is largely specialist.

Successful models target individual facilities and tasks. Yet accelerators reuse related RF, magnet, vacuum, timing, diagnostic, and control systems—an opportunity to learn once and transfer.

02

Data challenge

Operational telemetry has no single fixed schema.

Sensor sets and ordering, sampling rates, window lengths, physical units, noise, and operating conditions vary across facilities, subsystems, and acquisitions.

03

Shared backbone

Pretrain once across sensors, facilities, and time.

SensOFormer encodes channels as a set, models their temporal context, and learns by reconstructing clean values from masked, noisy telemetry.

04

Evidence

One representation transfers to four operational tasks.

Across five facilities, SensOFormer leads 21 of 24 zero-shot imputation settings, passes 20 of 21 early-fault tests, and reaches 84.80% / 84.01% cavity / fault accuracy.

The challenge

Why standard fixed-channel assumptions break down in facility-scale telemetry.

Accelerator data are not one tidy multivariate time series.

A channel’s meaning comes from its sensor, physical location, and subsystem—not its column index. The observed channels can change between acquisitions, while each subsystem evolves on its own time scale.

01

Variable sensor availability

Channels range from one to 1,754 in the LANSCE corpus, with no fixed global layout required by the model.

02

Multiple temporal scales

Fast RF and beam events coexist with slower vacuum, cooling, and power-supply behavior.

03

Missing and noisy measurements

Isolated dropouts, missing intervals, unavailable channels, and structured gaps all occur in operation.

04

Operating drift

Configurations, calibration, component aging, and operating conditions shift the sensor distribution over time.

The method

Set-aware in channels.
Attentive through time.

SensOFormer combines permutation-invariant sensor encoding, latent temporal processing, and permutation-equivariant reconstruction. The result is one architecture that accommodates different channel sets and sequence structures without channel-dimension padding.

01

Mask + denoise

Corrupt visible telemetry with point, time, channel, or block masks plus sampled Gaussian noise.

02

Encode the set

Learnable latent queries cross-attend to channel tokens, independent of their input ordering.

03

Model temporal context

A latent transformer processes the compact sequence to learn structure across time.

04

Decode or adapt

Reconstruct queried channels or attach a lightweight head for binary and multiclass tasks.

Figure 02

SensOFormer architecture and transfer strategy

Enlarge figure
The paper's architecture figure showing four masked-denoising strategies, a permutation-invariant set encoder, latent transformer, permutation-equivariant set decoder, and downstream reconstruction and classification heads.
Paper Fig. 2

The encoder turns the channels observed at each time step into a compact latent; the processor models time; the decoder preserves the identities of queried output channels. The pretrained encoder–processor backbone is reused for downstream tasks.

16

latent slots
in the set encoder

32

memory slots
in the set decoder

2.38–80.54M

parameters
across five evaluated scales

1,024

time steps
per pretraining window

Scaling behavior

More capacity and more data lower reconstruction loss.

The scaling study asks whether SensOFormer continues to learn better representations as the backbone and pretraining corpus grow. Across five model sizes and four data fractions, validation masked-reconstruction loss decreases along both axes.

The paper summarizes this behavior with a separable power law: one term captures limits from model capacity, while a second captures limits from available pretraining data.

General law · Eq. 7

N is parameter count, D is pretraining-data size, and L∞ is the fitted asymptotic loss.

SensOFormer fit · Eq. 8

The fit uses raw counts for N and D and has RMSE 4 × 10−4 over the observed configurations.

2.38–80.54Mparameters across five model variants
12.5–100%of the pretraining corpus across four data scales

Interpretation. This is evidence of improvement within the evaluated grid—not a guarantee that the same power law extrapolates beyond it. The practical message is to grow model capacity and accelerator data together.

Figure 03

Scaling model and data together

Enlarge figure
Fitted validation-loss contours across five SensOFormer model sizes and four pretraining-data sizes, with loss decreasing toward the largest evaluated model and dataset.
Paper Fig. 3

Red markers show evaluated model–data configurations; the yellow marker is the best fitted point within the measured grid.

Pretraining data

Heterogeneity is the training signal.

SensOFormer learns from five accelerator data families without forcing them into one global channel schema. Each source keeps its own sensors, sequence structure, and operational context.

Five-source corpus

  • 01LANSCEOperational telemetry · LANL4,250 records · 1–1,754 ch.
  • 02LHC-BLBeam-loss monitoring · CERN2 long sequences · 66 ch.
  • 03HVCM-SNSPower-electronics waveforms · ORNL10,000 samples · 12 ch.
  • 04RFS-LCLSRF-station signals · SLAC9,841 records · 9 ch.
  • 05SRF-CEBAFSuperconducting-RF events · JLab2,375 events · 192 steps

From sequence to training example

  1. Split first

    Original sequences are divided 90/10 before windowing, so overlapping windows cannot leak across train and validation sets.

  2. Window

    Each sequence becomes a 1,024-step window with stride 256; channel counts remain dataset-specific with no channel padding.

  3. Normalize

    Visible values are scaled per window and per channel, using no statistics from the masked targets.

  4. Denoise

    Noise is added only to visible measurements, teaching the model to recover structure instead of copying inputs.

Mask bank · one pattern sampled per window

Point-wiseHide 15% of individual values
Time-wiseHide 75% of channels at one time
Channel-wiseHide 60% of one sensor’s time steps
Block-wiseHide a contiguous ≈15% time–channel region

Together, the masks reproduce isolated dropouts, missing time slices, intermittent sensors, and structured gaps seen in accelerator telemetry.

Zero-shot reconstruction

Seeing what the model restores—not only its score.

The paper masks 15% of held-out sensor values and asks the pretrained model to recover them from temporal and cross-channel context. These appendix examples span four accelerator facilities and require no task-specific fine-tuning.

How to read the figures

Blue traces are the recorded sensor values. Red marks identify the values hidden from the model; green circles show SensOFormer-B’s predictions at exactly those locations.

Point-wise mask

Individual measurements disappear across the window, mimicking scattered dropouts.

Block-wise mask

A contiguous time–channel region disappears, creating a structured gap.

21 / 24

evaluated dataset–mask settings were led by SensOFormer-B; the smaller SensOFormer-S led the remaining three.

Examples in the sequence

LANSCE · LHC at CERN · SNS at Oak Ridge · LCLS at SLAC

Appendix figures 4–12

Nine views of missing-data recovery

Enlarge view

Facility

Mask

LANSCE 2019–2024 point-wise imputation with 15 percent of sensor values hidden; blue is ground truth, red is hidden data, and green is the model prediction.

LANSCE · 2019–2024Point-wise mask · Paper Fig. 4

01 / 09
15% hidden

Choose a facility and mask type to inspect the reported examples without waiting for an animation. The explorer includes every qualitative imputation figure in the appendix.

Supervised transfer · CEBAF

Fault identification with fewer expert labels.

Once a faulty event is detected, operators still need to identify what failed. SensOFormer-XTi transfers its pretrained sensor representation to eight CEBAF fault classes using a lightweight temporal classification head.

Paper figure 1

Fault accuracy as labeled data increases

Enlarge figure
Fault-identification accuracy rises from about 66 percent with 10 percent of labeled training data to 84.01 percent with all training data. At 25 percent of the data, it already exceeds the prior 74.10-percent benchmark.
8 fault classes

The blue curve is SensOFormer-XTi test accuracy; the orange dashed line marks the 74.10% prior deep-learning benchmark. The held-out test set remains fixed as the labeled training fraction changes.

The label-efficiency result

With only one quarter of the labeled fault examples, the transferred model already moves above the previous benchmark. More labels continue to help, but pretraining makes the scarce-data regime useful much earlier.

25%of labeled training data is enough to exceed the prior benchmark
84.01%fault-identification accuracy with the full training set
+9.91 ptsimprovement over the 74.10% fault benchmark at full data

Downstream tasks

One representation, three modes of use.

The pretrained encoder and temporal processor form the shared backbone. SensOFormer either reuses its reconstruction decoder directly or adds a compact task-specific head.

01

Zero-shot · reconstruction

Missing-data imputation

Recover masked sensor values from temporal and cross-channel context across six held-out test settings. No downstream labels or task-specific training are required.

21 / 24dataset–mask settings led by the B model
02

Frozen backbone · binary head

Anomaly detection

A 0.22M-parameter temporal CNN head learns normal-versus-faulty HVCM-SNS behavior while the pretrained XTi encoder and processor remain frozen.

20 / 21induced early-fault tests passed · 95.2%
03

Pretrained backbone · multiclass heads

Cavity & fault identification

Separate lightweight heads identify the first faulting CEBAF cavity across 9 classes and the fault type across 8 classes. The fault benchmark is surpassed with only 25% of the labeled training set.

84.80% / 84.01%cavity / fault test accuracy

Transfer principle: preserve the learned sensor representation; change only the output interface needed by the operational task.

Figure