Scientific Simulations with WitEngine

Scientific computing lives at the intersection of compute power and time. A climate model that takes 6 months on one machine isn't just slow—it's useless for decision-making. A parameter sweep across 10,000 configurations isn't ambitious—it's the minimum for meaningful results.

WitEngine transforms these computational challenges from infrastructure problems into workflow definitions. You describe the simulation; WitEngine handles distribution across whatever hardware you can access—from a rack of lab workstations to a global compute mesh.

The Core Use Cases

Scientific simulations fall into two dominant patterns: Monte Carlo methods (random sampling for estimation) and parameter sweeps (systematic exploration of configuration space). Both are embarrassingly parallel—and both benefit enormously from distribution.

Monte Carlo Simulations

Monte Carlo methods use random sampling to approximate solutions that would be intractable analytically. The accuracy scales with sample count: more samples, tighter confidence intervals.

Application	Typical Sample Count	Single Core	100 Cores
Option pricing	10 million	5 minutes	3 seconds
Value at Risk (VaR)	100 million	1 hour	36 seconds
Particle transport	1 billion	2 days	30 minutes
Protein folding	10 billion	3 weeks	5 hours

The math is straightforward: divide samples into batches, run batches in parallel, aggregate statistics. The engineering is where WitEngine helps.

~ Monte Carlo option pricing ~
Int:totalPaths = 10000000;      ~ 10 million paths ~
Int:pathsPerBatch = 100000;     ~ 100K per batch = 100 tasks ~

MonteCarloTaskCollection:tasks = [];
Int:batchIndex = 0;

While(batchIndex < Int.Divide(totalPaths, pathsPerBatch))
{
    ~ Each batch gets unique reproducible seed ~
    Long:seed = Long.Add(baseSeed, Long.Multiply(batchIndex, 1000000));
    
    MonteCarloTask:task = MonteCarloTask.Create(simulation, pathsPerBatch, seed);
    task = MonteCarloTask.SetParameter(task, "spot", spotPrice);
    task = MonteCarloTask.SetParameter(task, "strike", strikePrice);
    task = MonteCarloTask.SetParameter(task, "volatility", volatility);
    
    tasks = Collection.Add(tasks, task);
    batchIndex = Int.Add(batchIndex, 1);
}

~ Distribute across available nodes ~
ProcessingOptions:opts = ProcessingOptions.Create("Balanced");

MonteCarloResultCollection:results = 
    Grid.ForEach(task in tasks, opts) => MonteCarlo.SimulateBatch(task);

~ Aggregate: sum payoffs, compute mean, calculate standard error ~

Parameter Sweeps

Parameter sweeps explore how system behavior changes across configuration space. Climate models vary CO₂ concentrations. Drug simulations test dosage ranges. Engineering models stress-test design parameters.

Each configuration is independent—perfect for distribution:

Domain	Parameters	Configurations	Single Machine	Distributed
Climate modeling	5 params × 20 values each	3.2 million	8 years	3 days
Drug dosing	3 params × 100 values	1 million	2 years	18 hours
CFD optimization	8 params × 10 values	100 million	Impossible	2 weeks
Material science	4 params × 50 values	6.25 million	15 years	5 days

Reproducibility: The Scientific Imperative

In science, a result you can't reproduce isn't a result—it's an anecdote. WitEngine's architecture supports reproducibility at every level.

Explicit Seeding

Random number generators must produce identical sequences given identical seeds. WitEngine enforces this through explicit seed management:

~ Base seed for entire experiment ~
Long:experimentSeed = 20240115;  ~ Date-based for traceability ~

~ Each task derives its seed deterministically ~
ForEach(configIndex in Range(0, numConfigurations))
{
    Long:taskSeed = Long.Add(experimentSeed, Long.Multiply(configIndex, 1000000));
    
    SimulationTask:task = SimulationTask.Create(config, taskSeed);
    tasks = Collection.Add(tasks, task);
}

Re-run the job with the same base seed → identical results, regardless of which nodes execute which tasks.

Run Metadata

Every execution captures complete provenance:

csharp

[MemoryPackable]
public partial class SimulationRunMetadata
{
    public string RunId { get; set; }              // Unique identifier
    public DateTime StartTime { get; set; }        // When job started
    public DateTime EndTime { get; set; }          // When job completed
    public string ScriptVersion { get; set; }      // Git commit hash
    public string ModuleVersion { get; set; }      // Simulation module version
    public long BaseSeed { get; set; }             // Random seed
    public Dictionary<string, string> Parameters { get; set; }  // All input parameters
    public int TotalTasks { get; set; }            // Task count
    public int SuccessfulTasks { get; set; }       // Completed successfully
    public List<string> NodeIds { get; set; }      // Which nodes participated
}

Artifact Management

Simulation outputs need systematic organization:

~ Structured output with full traceability ~
String:runId = Guid.New();
String:outputBase = String.Concat("/results/", runId);

File.EnsureDirectory(String.Concat(outputBase, "/raw"));
File.EnsureDirectory(String.Concat(outputBase, "/aggregated"));
File.EnsureDirectory(String.Concat(outputBase, "/metadata"));

~ Save run configuration ~
Simulation.SaveMetadata(runMetadata, String.Concat(outputBase, "/metadata/run.json"));

~ Each task saves its raw output ~
Grid.ForEach(task in tasks, opts) => Simulation.RunWithArtifacts(task, outputBase);

~ Aggregate and save final results ~
Simulation.AggregateResults(outputBase);
Simulation.SaveSummary(outputBase);

Output structure:

/results/a1b2c3d4-e5f6-7890-abcd-ef1234567890/
├── metadata/
│   ├── run.json              # Complete run configuration
│   ├── parameters.json       # Input parameters
│   └── nodes.json            # Participating nodes and their specs
├── raw/
│   ├── task_0000.dat         # Raw output from task 0
│   ├── task_0001.dat         # Raw output from task 1
│   └── ...
└── aggregated/
    ├── summary.json          # Aggregated statistics
    ├── distributions.csv     # Full distribution data
    └── figures/              # Generated visualizations

Version Control Integration

Pin exact versions for reproducibility:

~ Record versions at job start ~
RunVersions:versions = RunVersions.Create();
versions = RunVersions.SetScript(versions, Git.CurrentCommit());
versions = RunVersions.SetModule(versions, Module.Version("MonteCarlo"));
versions = RunVersions.SetEngine(versions, WitEngine.Version());

~ Validate versions match expected ~
If(versions.Module != expectedModuleVersion)
{
    Trace("WARNING: Module version mismatch");
    Trace("  Expected:", expectedModuleVersion);
    Trace("  Found:", versions.Module);
}

Heterogeneous Nodes: Real-World Hardware

Research computing environments are messy. Your cluster includes:

The department's 5-year-old compute server (64 cores, 256GB RAM)
Graduate students' workstations (8-16 cores, 32GB RAM)
That one machine with the expensive GPU someone bought for a grant
Cloud burst capacity when deadlines loom

WitEngine handles this heterogeneity through capability matching and benchmark-based allocation.

CPU Requirements

Some simulations need specific CPU features:

~ Require AVX-512 for vectorized computation ~
ProcessingOptions:opts = ProcessingOptions.Create("Balanced");
opts = ProcessingOptions.SetRequirement(opts, "CPU_CORES", "8");
opts = ProcessingOptions.SetRequirement(opts, "CPU_ARCH", "X64");

~ Only nodes meeting requirements receive tasks ~

The engine filters nodes based on declared requirements:

Activity declares: "I need 8+ cores, X64 architecture"

Node A: 64 cores, X64  ✓ Compatible (receives more work due to benchmark)
Node B: 4 cores, X64   ✗ Not enough cores
Node C: 16 cores, ARM  ✗ Wrong architecture  
Node D: 32 cores, X64  ✓ Compatible

Memory Constraints

Large simulations need nodes with sufficient RAM:

~ Memory-intensive matrix operations ~
ProcessingOptions:opts = ProcessingOptions.Create("Queued");
opts = ProcessingOptions.SetRequirement(opts, "RAM", "64GB");

~ For variable memory needs, estimate per-task ~
ForEach(matrix in matrices)
{
    Long:estimatedMemory = Long.Multiply(matrix.Rows, Long.Multiply(matrix.Cols, 8));
    
    If(Long.IsGreater(estimatedMemory, 32000000000))  ~ >32GB ~
    {
        task = MatrixTask.SetRequirement(task, "RAM", "128GB");
    }
    Else
    {
        task = MatrixTask.SetRequirement(task, "RAM", "32GB");
    }
}

Benchmark-Based Allocation

Hardware specs don't tell the whole story. A 16-core Xeon might outperform a 32-core older processor for your specific workload. WitEngine's benchmark system measures actual performance:

Benchmark results for eigenvalue decomposition:

Node A (Xeon Gold 6248R, 24 cores):     1.0 (baseline)
Node B (Ryzen 9 5950X, 16 cores):       0.85
Node C (Xeon E5-2680 v4, 28 cores):     0.45  (older, slower per-core)
Node D (EPYC 7742, 64 cores):           2.1

Task allocation (1000 tasks):
  Node A: 227 tasks (22.7%)
  Node B: 193 tasks (19.3%)
  Node C: 102 tasks (10.2%)
  Node D: 478 tasks (47.8%)

All nodes finish together → maximum throughput

Custom Capabilities

Domain-specific requirements beyond standard hardware:

csharp

// Custom property for licensed software
public class RequiresMatlabAttribute : RequirementAttribute
{
    public string MinVersion { get; set; }
    
    public override bool IsSatisfiedBy(IWitCapabilities caps)
    {
        return caps.CustomProperties.TryGetValue("matlab-version", out var version)
               && VersionCompare(version, MinVersion) >= 0;
    }
}

// Usage
[RequiresMatlab(MinVersion = "R2023a")]
public class MatlabSimulationActivity : WitActivityFunction { }

Example: 10,000-Job Parameter Sweep

Let's walk through a complete parameter sweep—the kind of job that separates "we could explore this" from "we actually explored this."

The Scenario

You're optimizing a chemical process simulation. Four parameters, each with a meaningful range:

Parameter	Range	Steps	Values
Temperature	300-500 K	20	300, 310, 320, ... 500
Pressure	1-10 atm	10	1, 2, 3, ... 10
Catalyst concentration	0.01-0.10 M	10	0.01, 0.02, ... 0.10
Reaction time	1-50 hours	50	1, 2, 3, ... 50

Total configurations: 20 × 10 × 10 × 50 = 100,000 jobs

Each simulation takes ~30 seconds. Sequential execution: 35 days. Distributed across 100 nodes: 8 hours.

The Script

~ ========================================================================
~ CHEMICAL PROCESS PARAMETER SWEEP
~ 100,000 configurations across 4 parameters
~ ========================================================================

~ Parameter ranges ~
DoubleCollection:temperatures = Double.Range(300, 500, 20);    ~ 20 values ~
DoubleCollection:pressures = Double.Range(1, 10, 10);          ~ 10 values ~
DoubleCollection:catalysts = Double.Range(0.01, 0.10, 10);     ~ 10 values ~
DoubleCollection:reactionTimes = Double.Range(1, 50, 50);      ~ 50 values ~

~ Calculate total configurations ~
Int:totalConfigs = Int.Multiply(
    Int.Multiply(Collection.Count(temperatures), Collection.Count(pressures)),
    Int.Multiply(Collection.Count(catalysts), Collection.Count(reactionTimes))
);

Trace("Parameter Sweep Configuration:");
Trace("  Temperatures:", Collection.Count(temperatures), "values");
Trace("  Pressures:", Collection.Count(pressures), "values");
Trace("  Catalysts:", Collection.Count(catalysts), "values");
Trace("  Reaction times:", Collection.Count(reactionTimes), "values");
Trace("  Total configurations:", String.Format("{0:N0}", totalConfigs));
Trace("");

~ Setup reproducibility ~
Long:baseSeed = 20240115;
String:runId = Guid.New();
String:outputPath = String.Concat("/results/sweep_", runId);

File.EnsureDirectory(outputPath);

~ Create all tasks ~
Trace("Creating simulation tasks...");
DateTime:setupStart = DateTime.Now();

SimulationTaskCollection:tasks = [];
Int:taskIndex = 0;

ForEach(temp in temperatures)
{
    ForEach(pressure in pressures)
    {
        ForEach(catalyst in catalysts)
        {
            ForEach(time in reactionTimes)
            {
                ~ Deterministic seed for reproducibility ~
                Long:taskSeed = Long.Add(baseSeed, taskIndex);
                
                SimulationTask:task = SimulationTask.Create(taskIndex, taskSeed);
                task = SimulationTask.SetParameter(task, "temperature", temp);
                task = SimulationTask.SetParameter(task, "pressure", pressure);
                task = SimulationTask.SetParameter(task, "catalyst", catalyst);
                task = SimulationTask.SetParameter(task, "reactionTime", time);
                task = SimulationTask.SetOutputPath(task, outputPath);
                
                tasks = Collection.Add(tasks, task);
                taskIndex = Int.Add(taskIndex, 1);
            }
        }
    }
}

DateTime:setupEnd = DateTime.Now();
Trace("Created", Collection.Count(tasks), "tasks in", 
      DateTime.DiffSeconds(setupStart, setupEnd), "seconds");
Trace("");

~ Configure distribution ~
ProcessingOptions:opts = ProcessingOptions.Create("Balanced");
opts = ProcessingOptions.SetRequirement(opts, "CPU_CORES", "4");
opts = ProcessingOptions.SetRequirement(opts, "RAM", "8GB");
opts = ProcessingOptions.SetProgressReporting(opts, true);

~ Save run metadata before execution ~
RunMetadata:metadata = RunMetadata.Create(runId, baseSeed, totalConfigs);
metadata = RunMetadata.SetParameters(metadata, {
    "temperatures": temperatures,
    "pressures": pressures,
    "catalysts": catalysts,
    "reactionTimes": reactionTimes
});
Simulation.SaveMetadata(metadata, String.Concat(outputPath, "/metadata.json"));

~ Execute distributed sweep ~
Trace("Starting distributed parameter sweep...");
Trace("Estimated time: ~8 hours on 100 nodes");
Trace("");

DateTime:startTime = DateTime.Now();

SimulationResultCollection:results = 
    Grid.ForEach(task in tasks, opts) => ChemSim.RunSimulation(task);

DateTime:endTime = DateTime.Now();
Double:totalTime = DateTime.DiffSeconds(startTime, endTime);

~ Analyze results ~
Int:successCount = 0;
Int:failedCount = 0;
Double:maxYield = 0;
SimulationResult:bestResult = null;

ForEach(result in results)
{
    If(result.Success == true)
    {
        successCount = Int.Add(successCount, 1);
        
        If(Double.IsGreater(result.Yield, maxYield) == true)
        {
            maxYield = result.Yield;
            bestResult = result;
        }
    }
    Else
    {
        failedCount = Int.Add(failedCount, 1);
    }
}

~ Report ~
Trace("========================================");
Trace("PARAMETER SWEEP COMPLETE");
Trace("========================================");
Trace("");
Trace("Execution:");
Trace("  Total time:", String.Format("{0:F1}", Double.Divide(totalTime, 3600)), "hours");
Trace("  Configurations tested:", successCount);
Trace("  Failed:", failedCount);
Trace("  Throughput:", String.Format("{0:F1}", Double.Divide(successCount, totalTime)), "configs/sec");
Trace("");
Trace("Best Result:");
Trace("  Temperature:", bestResult.Temperature, "K");
Trace("  Pressure:", bestResult.Pressure, "atm");
Trace("  Catalyst:", bestResult.Catalyst, "M");
Trace("  Reaction time:", bestResult.ReactionTime, "hours");
Trace("  Yield:", String.Format("{0:P2}", bestResult.Yield));
Trace("");

~ Save aggregated results ~
Simulation.SaveResults(results, String.Concat(outputPath, "/results.json"));
Simulation.GenerateHeatmaps(results, String.Concat(outputPath, "/heatmaps/"));
Simulation.ExportCSV(results, String.Concat(outputPath, "/results.csv"));

Trace("Results saved to:", outputPath);

What You Get

After 8 hours instead of 35 days:

/results/sweep_a1b2c3d4/
├── metadata.json           # Complete reproducibility info
├── results.json            # All 100,000 results
├── results.csv             # For analysis in R/Python/Excel
├── heatmaps/
│   ├── temp_vs_pressure.png
│   ├── catalyst_vs_time.png
│   └── ...
└── raw/
    ├── task_00000.dat
    ├── task_00001.dat
    └── ... (100,000 files)

Scaling to WitCloud and OmnibusCloud

A departmental cluster handles routine simulations. But what about the ambitious project—the one that needs 10× more compute for a conference deadline?

Local WitCloud: Your Organization's Hidden Supercomputer

WitEngine is the core of WitCloud, a platform that unifies idle resources across your organization.

The university scenario:

Physics department: 50 workstations
Chemistry labs: 30 workstations  
Engineering building: 80 workstations
Graduate student machines: 200 laptops

Daytime: Researchers using 5-10% of CPU capacity
Nighttime: 360 machines sitting idle

With Local WitCloud:
  → Submit parameter sweep at 6 PM
  → 360 nodes process overnight
  → Results ready by 8 AM
  → Zero additional hardware cost

Multi-campus deployments can leverage time zones:

Main campus (Boston, UTC-5):     200 machines, idle 8 PM - 8 AM
Satellite campus (London, UTC+0): 100 machines, idle 8 PM - 8 AM
Research station (Tokyo, UTC+9):   50 machines, idle 8 PM - 8 AM

At any moment: 150-200 machines available
Result: Near-continuous compute capacity

OmnibusCloud: Global Compute on Demand

OmnibusCloud is the public instance of WitCloud—a worldwide mesh of compute resources.

For scientific computing, this means:

Scenario	Local Cluster	Local WitCloud	OmnibusCloud
10K parameter sweep	3 hours	30 minutes	2 minutes
100K parameter sweep	30 hours	5 hours	20 minutes
1M Monte Carlo paths	2 hours	15 minutes	1 minute
1B Monte Carlo paths	8 days	20 hours	2 hours

The script doesn't change:

~ Same script, any scale ~
ProcessingOptions:opts = ProcessingOptions.Create("Balanced");
opts = ProcessingOptions.SetRequirement(opts, "CPU_CORES", "4");
opts = ProcessingOptions.SetRequirement(opts, "RAM", "8GB");

~ Local cluster, WitCloud, or OmnibusCloud—transparent to script ~
SimulationResultCollection:results = 
    Grid.ForEach(task in tasks, opts) => Simulation.Run(task);

Hybrid Execution

Run sensitive preliminary analysis locally; burst to OmnibusCloud for production sweeps:

Typical research workflow:

Development phase:
  → Local workstation (1 node)
  → Test with 100 configurations
  → Debug, validate, iterate

Validation phase:
  → Department cluster (20 nodes)
  → Run 1,000 configurations
  → Verify scaling behavior

Production phase:
  → OmnibusCloud (1,000+ nodes)
  → Full 100,000 configuration sweep
  → Results in hours, not months

Reproducibility Across Platforms

Same seed → same results, regardless of where tasks execute:

~ Run on local cluster ~
Long:baseSeed = 20240115;
results_local = RunSweep(parameters, baseSeed, localClusterOpts);

~ Re-run on OmnibusCloud ~
results_cloud = RunSweep(parameters, baseSeed, omnibusOpts);

~ Results are identical (within floating-point precision) ~
Assert(Results.AreEquivalent(results_local, results_cloud));

The orchestrator tracks which nodes executed which tasks, but the mathematical results depend only on inputs and seeds—not on execution environment.

Summary

WitEngine transforms scientific simulation from an infrastructure challenge into a workflow definition:

Challenge	WitEngine Solution
Monte Carlo at scale	Batch distribution with automatic aggregation
Parameter sweeps	Systematic configuration generation and parallel execution
Reproducibility	Explicit seeding, run metadata, artifact management
Heterogeneous hardware	Capability requirements, benchmark-based allocation
Burst capacity	Seamless scaling to WitCloud and OmnibusCloud

The math doesn't change. The algorithms don't change. What changes is the time from question to answer—from months to hours, from "we could explore this" to "we explored this."

For scientific computing, that's the difference that matters.