Design Philosophy

WitEngine's architecture reflects a set of deliberate design choices. Understanding these choices helps you use the framework effectively and appreciate why certain patterns are encouraged while others are discouraged.

The Central Principle: Scripts Orchestrate, Modules Compute

This single principle shapes everything in WitEngine:

Why This Separation?

Consider the alternative: a powerful scripting language that can do anything. This is what most systems provide. The problems:

❌ Powerful scripts lead to:
   ├── Complex logic embedded in scripts (hard to maintain)
   ├── Performance-critical code in interpreted layer (slow)
   ├── Hidden dependencies between operations (can't distribute)
   ├── Security risks from arbitrary code execution
   └── Scripts that only the author understands

WitEngine takes the opposite approach:

✅ Limited scripts lead to:
   ├── Clear workflow visible at a glance
   ├── Complex logic in compiled, tested modules
   ├── Explicit data flow (distribution-friendly)
   ├── Sandboxed execution by design
   └── Scripts anyone can read and audit

A Practical Example

Imagine you need to apply a complex image filter to 10,000 photos.

In a traditional system, you might write the filter algorithm directly in your script. This creates problems:

• The algorithm runs in an interpreter (slow)
• The script becomes hundreds of lines of image manipulation code
• You can't easily distribute because the engine doesn't understand what the code does

In WitEngine, you write a module that implements the filter in optimized C#. Your script becomes:

ImageCollection:enhanced = Grid.ForEach(img in photos) 
    => ImageFilter.Apply(img, "sharpen");

The script says what you want (apply filter to photos). The module handles how (the actual algorithm). The engine can distribute this because it understands the pattern: "apply this operation to each item."

Why the Language Is Intentionally Limited

WitEngine's scripting language lacks features that most developers expect:

The Benefits of Constraints

1. Transparent Computation Graph

Because every operation is an explicit activity call, WitEngine can build a complete picture of what your script does. This enables:

• Static analysis before execution
• Automatic parallelization opportunities
• Precise resource estimation
• Intelligent distribution

2. Compile-Time Safety

With no complex expressions, the parser can validate everything:

Int:x = 10;
Int:y = 20;
Int:sum = Int.Add(x, y);     ~ ✓ Validated at parse time ~

String:name = Int.Add(x, y); ~ ✗ Type error caught immediately ~

3. Auditable Workflows

A WitEngine script reads like a checklist:

Job:ProcessBatch(DataCollection:input)
{
    ~ 1. Validate input ~
    ValidationResult:check = Validator.Check(input);
    
    ~ 2. Transform data ~
    DataCollection:transformed = Transform.Apply(input, "normalize");
    
    ~ 3. Process in parallel across cluster ~
    ResultCollection:results = Grid.ForEach(item in transformed)
        => Processor.Analyze(item);
    
    ~ 4. Aggregate and return ~
    Summary:final = Aggregator.Summarize(results);
    Return(final);
}

Anyone can understand this workflow without knowing implementation details.

4. Forced Modularity

When you can't write complex logic in scripts, you're forced to put it in modules. This leads to:

• Better tested code (modules have unit tests)
• Reusable components (modules work across scripts)
• Clear interfaces (activities have defined inputs/outputs)

Designed for Distribution

Every aspect of WitEngine is designed with distributed execution in mind.

Stateless Operations

Activities don't share state. Each activity call:

• Receives all inputs explicitly as parameters
• Produces outputs that are stored in the variable pool
• Has no side effects on other activities

This statelessness means activities can run anywhere — on the local machine, on a remote node, or distributed across many nodes.

Serialization-First

All data types in WitEngine must be serializable. This isn't an afterthought — it's a fundamental requirement. When you create a custom type, you make it serializable from the start:

[MemoryPackable]
public partial class Temperature
{
    public double Value { get; set; }
    public string Unit { get; set; }
}

This ensures that any variable can be sent to any node at any time.

Explicit Data Dependencies

Because scripts use explicit variable references, the engine knows exactly which data each activity needs:

Int:a = 10;
Int:b = 20;
Int:sum = Int.Add(a, b);    ~ Needs: a, b ~
Int:doubled = Int.Multiply(sum, 2);  ~ Needs: sum ~

This dependency information enables:

• Parallel execution of independent operations
• Minimal data transfer (send only what's needed)
• Intelligent scheduling

Type Safety as a Foundation

WitEngine is strongly typed throughout. Types aren't just documentation — they're enforced.

Parse-Time Validation

Type errors are caught when the script is compiled, not when it runs:

~ These fail at parse time, not runtime: ~
Int:count = "hello";           ~ Type mismatch ~
Matrix:m = Vector.Create(1,2); ~ Vector is not Matrix ~
Int:result = Process(data);    ~ If Process returns String ~

Why This Matters for Distribution

In a distributed system, runtime errors are expensive:

• Work may have already been sent to nodes
• Partial results need to be handled
• Debugging is harder (which node failed? why?)

By catching errors at parse time, WitEngine ensures that if a script compiles, the type flow is valid. You won't get type errors on node #47 after an hour of processing.

Generic Operations with Type Preservation

Collections maintain their element types:

IntCollection:numbers = [1, 2, 3, 4, 5];
Int:first = Collection.First(numbers);     ~ Returns Int ~
Int:sum = Collection.Sum(numbers);         ~ Returns Int ~

StringCollection:names = ["Alice", "Bob"];
String:first = Collection.First(names);    ~ Returns String ~

The engine tracks types through all operations, ensuring consistency.

Plugin Architecture: Composition Over Configuration

WitEngine's plugin system follows the principle of composition over configuration.

Everything Is a Plugin

Even built-in functionality comes from plugins (Controllers):

There's no "core language" with special privileges. If you don't load the Variables controller, you don't have Int. This means:

• You can replace built-in behavior
• You can run minimal configurations
• Custom types are first-class citizens

Explicit Dependencies

Plugins declare their dependencies:

[WitPluginManifest("Physics", Version = "1.0.0")]
[WitPluginDependency("Variables", MinimumVersion = "1.0.0")]
[WitPluginDependency("Matrices", MinimumVersion = "1.0.0")]
public class PhysicsController : IWitControllerHost

This creates a clear dependency graph:

• The engine loads plugins in the correct order
• Version conflicts are detected before runtime
• You can reason about what capabilities are available

Capability Declaration

Plugins don't just provide features — they declare requirements:

[RequiresGpu(Features = GpuFeatures.Cuda)]
public class WitActivityTrainModel : WitActivityTransform

The engine uses these declarations to:

• Filter nodes that can't run certain activities
• Provide clear error messages when requirements aren't met
• Enable capability-aware distribution

Predictability Over Magic

WitEngine favors explicit, predictable behavior over "magic" that's hard to understand.

No Implicit Conversions

Int:x = 10;
Double:y = x;          ~ ❌ No implicit conversion ~
Double:y = Int.ToDouble(x);  ~ ✓ Explicit conversion ~

No Operator Overloading

Matrix:a = ...;
Matrix:b = ...;
Matrix:c = a + b;              ~ ❌ No operator overloading ~
Matrix:c = Matrix.Add(a, b);   ~ ✓ Explicit operation ~

No Hidden Execution

What you write is what executes. There are no:

• Lazy evaluation surprises
• Automatic caching
• Implicit parallelization
• Background operations

If you want parallelization, you write Parallel.ForEach or Grid.ForEach. The script shows exactly what happens.

Security Through Isolation

Scripts run in a sandboxed environment with no direct access to:

Why This Matters

In a distributed system, scripts might run on machines you don't fully control. Sandboxing ensures:

• A malicious script can't damage nodes
• Scripts can't exfiltrate data except through defined channels
• Resource access is auditable (you know which activities can do what)

Capability-Based Security

Access is granted through plugins. If you want file access:

[WitPluginManifest("FileSystem", Version = "1.0.0")]
public class FileSystemController : IWitControllerHost
{
    // Registers File.Read, File.Write, etc.
}

If you don't load this plugin, file access doesn't exist. This makes security auditing straightforward: check which plugins are loaded.

Summary: The WitEngine Way

These principles work together to create a system where:

• Scripts are simple and auditable
• Complex logic is properly encapsulated
• Distribution happens transparently
• Errors are caught early
• Security is built-in, not bolted-on

Understanding these principles helps you work with WitEngine rather than against it. When something feels awkward, it's usually because you're trying to put logic in the wrong layer.