EasyHybrid Example: Synthetic Data Analysis

This example demonstrates how to use EasyHybrid to train a hybrid model on synthetic data for respiration modeling with Q10 temperature sensitivity.

using EasyHybrid
using Metal
# using CUDA
using Lux

Data Loading and Preprocessing

Load synthetic dataset from GitHub into DataFrame

df = load_timeseries_netcdf("https://github.com/bask0/q10hybrid/raw/master/data/Synthetic4BookChap.nc");

Select a subset of data for faster execution

df = df[1:5000, :];
first(df, 5)

5×6 DataFrame

Row	time	sw_pot	dsw_pot	ta	reco	rb
	DateTime	Float64?	Float64?	Float64?	Float64?	Float64?
1	2003-01-01T00:15:00	109.817	115.595	2.1	0.844741	1.42522
2	2003-01-01T00:45:00	109.817	115.595	1.98	0.840641	1.42522
3	2003-01-01T01:15:00	109.817	115.595	1.89	0.837579	1.42522
4	2003-01-01T01:45:00	109.817	115.595	2.06	0.843372	1.42522
5	2003-01-01T02:15:00	109.817	115.595	2.09	0.844399	1.42522

Define the Physical Model

RbQ10 model: Respiration model with Q10 temperature sensitivity

Parameters:

• ta: air temperature [°C]

• Q10: temperature sensitivity factor [-]

• rb: basal respiration rate [μmol/m²/s]

• tref: reference temperature [°C] (default: 15.0)

function RbQ10(; ta, Q10, rb, tref = 15.0f0)
    reco = rb .* Q10 .^ (0.1f0 .* (ta .- tref))
    return (; reco, Q10, rb)
end

RbQ10 (generic function with 1 method)

Define Model Parameters

Parameter specification: (default, lower_bound, upper_bound)

parameters = (
    # Parameter name | Default | Lower | Upper      | Description
    rb = (3.0f0, 0.0f0, 13.0f0),  # Basal respiration [μmol/m²/s]
    Q10 = (2.0f0, 1.0f0, 4.0f0),   # Temperature sensitivity factor [-]
)

(rb = (3.0f0, 0.0f0, 13.0f0), Q10 = (2.0f0, 1.0f0, 4.0f0))

Configure Hybrid Model Components

Define input variables

forcing = [:ta]                    # Forcing variables (temperature)

1-element Vector{Symbol}:
 :ta

Target variable

target = [:reco]                   # Target variable (respiration)

1-element Vector{Symbol}:
 :reco

Parameter classification

global_param_names = [:Q10]        # Global parameters (same for all samples)
neural_param_names = [:rb]         # Neural network predicted parameters

1-element Vector{Symbol}:
 :rb

Single NN Hybrid Model Training

predictors_single_nn = [:sw_pot, :dsw_pot]   # Predictor variables (solar radiation, and its derivative)

small_nn_hybrid_model = constructHybridModel(
    predictors_single_nn,              # Input features
    forcing,                 # Forcing variables
    target,                  # Target variables
    RbQ10,                  # Process-based model function
    parameters,              # Parameter definitions
    neural_param_names,      # NN-predicted parameters
    global_param_names,      # Global parameters
    hidden_layers = [16, 16], # Neural network architecture
    activation = sigmoid,      # Activation function
    scale_nn_outputs = true, # Scale neural network outputs
    input_batchnorm = true   # Apply batch normalization to inputs
)

large_nn_hybrid_model = constructHybridModel(
    predictors_single_nn,              # Input features
    forcing,                 # Forcing variables
    target,                  # Target variables
    RbQ10,                  # Process-based model function
    parameters,              # Parameter definitions
    neural_param_names,      # NN-predicted parameters
    global_param_names,      # Global parameters
    hidden_layers = [1024, 512, 256, 128, 64], # Neural network architecture
    activation = sigmoid,      # Activation function
    scale_nn_outputs = true, # Scale neural network outputs
    input_batchnorm = true   # Apply batch normalization to inputs
)

Hybrid Model (Single NN)
Neural Network: 
  Chain(
      layer_1 = InputBatchNorm(
          layer = BatchNorm(2, affine=false, track_stats=true),
      ),
      layer_2 = Dense(2 => 1024, σ),                # 3_072 parameters
      layer_3 = Dense(1024 => 512, σ),              # 524_800 parameters
      layer_4 = Dense(512 => 256, σ),               # 131_328 parameters
      layer_5 = Dense(256 => 128, σ),               # 32_896 parameters
      layer_6 = Dense(128 => 64, σ),                # 8_256 parameters
      layer_7 = Dense(64 => 1),                     # 65 parameters
  )         # Total: 700_417 parameters,
            #        plus 5 states.
Configuration:
  mechanistic_model = RbQ10
  targets = [:reco]
  forcing = [:ta]
  neural_param_names = [:rb]
  predictors = [:sw_pot, :dsw_pot]
  global_param_names = [:Q10]
  fixed_param_names = Symbol[]
  scale_nn_outputs = true
  start_from_default = true
  config = (; hidden_layers = [1024, 512, 256, 128, 64], activation = σ, scale_nn_outputs = true, input_batchnorm = true, start_from_default = true,)

Parameters:
  ┌─────┬────────┬───────┬───────┬────────┐
  │     │  value │ lower │ upper │  scale │
  ├─────┼────────┼───────┼───────┼────────┤
  │  rb │     NN │   0.0 │  13.0 │ linear │
  │ Q10 │ Global │   1.0 │   4.0 │ linear │
  └─────┴────────┴───────┴───────┴────────┘
  value: number = fixed; NN/Global = estimated; Forcing = overridden by forcing data

train on DataFrame

Train the hybrid model

cfg = EasyHybrid.TrainConfig(
    nepochs = 20,
    batchsize = 64,
    opt = RMSProp(0.01),
    loss_types = [:mse, :nse],
    show_progress = false,
    keep_history = false, # set to true to keep per-epoch history, losses, predictions, etc.
    save_training = false, # Set to true to enable saving training history and checkpoints
)

using BenchmarkTools

gpu_small_nn() = tune(
    small_nn_hybrid_model, df, cfg;
    gdev = gpu_device(), model_name = "small_nn_gpu"
)

cpu_small_nn() = tune(
    small_nn_hybrid_model, df, cfg;
    gdev = cpu_device(), model_name = "small_nn_cpu"
)

gpu_large_nn() = tune(
    large_nn_hybrid_model, df, cfg;
    gdev = gpu_device(), model_name = "large_nn_gpu"
)

cpu_large_nn() = tune(
    large_nn_hybrid_model, df, cfg;
    gdev = cpu_device(), model_name = "large_nn_cpu"
)

cpu_large_nn (generic function with 1 method)

warm-up to pay compilation once

gpu_small_nn();
cpu_small_nn();
gpu_large_nn();
cpu_large_nn();

┌ Warning: Plotting enabled but keep_history is false. Plots will not be generated.
└ @ EasyHybrid ~/work/EasyHybrid.jl/EasyHybrid.jl/src/training/initialization.jl:14
┌ Warning: Could not determine number of GPU cores; some algorithms may not run optimally.
└ @ Metal ~/.julia/packages/Metal/TF981/src/utilities.jl:116
[ Info: Returning best model from epoch 7 with validation loss: 0.01729332
┌ Warning: Plotting enabled but keep_history is false. Plots will not be generated.
└ @ EasyHybrid ~/work/EasyHybrid.jl/EasyHybrid.jl/src/training/initialization.jl:14
[ Info: Returning best model from epoch 5 with validation loss: 0.0129249245
┌ Warning: Plotting enabled but keep_history is false. Plots will not be generated.
└ @ EasyHybrid ~/work/EasyHybrid.jl/EasyHybrid.jl/src/training/initialization.jl:14
[ Info: Returning best model from epoch 1 with validation loss: 0.10493142
┌ Warning: Plotting enabled but keep_history is false. Plots will not be generated.
└ @ EasyHybrid ~/work/EasyHybrid.jl/EasyHybrid.jl/src/training/initialization.jl:14
[ Info: Returning best model from epoch 2 with validation loss: 0.067143664

With Large NN CPU is slower than GPU

Large NN on GPU

@benchmark gpu_large_nn() samples = 4 evals = 1

BenchmarkTools.Trial: 1 sample with 1 evaluation per sample.
 Single result which took 30.966 s (0.28% GC) to evaluate,
 with a memory estimate of 1.46 GiB, over 11953105 allocations.

Large NN on CPU

@benchmark cpu_large_nn() samples = 4 evals = 1

BenchmarkTools.Trial: 1 sample with 1 evaluation per sample.
 Single result which took 5.356 s (3.50% GC) to evaluate,
 with a memory estimate of 6.11 GiB, over 2712486 allocations.

With Small NN CPU and GPU are on par

using BenchmarkTools

Small NN on GPU

@benchmark gpu_small_nn() samples = 4 evals = 1

BenchmarkTools.Trial: 1 sample with 1 evaluation per sample.
 Single result which took 20.279 s (0.00% GC) to evaluate,
 with a memory estimate of 422.60 MiB, over 8688398 allocations.

Small NN on CPU

@benchmark cpu_small_nn() samples = 4 evals = 1

BenchmarkTools.Trial: 4 samples with 1 evaluation per sample.
 Range (min … max):  299.033 ms … 336.040 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     329.891 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   323.714 ms ±  16.709 ms  ┊ GC (mean ± σ):  0.00% ± 0.00%

  █                                                 ██        █  
  █▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁██▁▁▁▁▁▁▁▁█ ▁
  299 ms           Histogram: frequency by time          336 ms <

 Memory estimate: 215.60 MiB, allocs estimate: 2154442.