EasyHybrid.jl

Documentation for EasyHybrid.jl.

EasyHybrid.EasyHybrid Module

julia

EasyHybrid.jl

EasyHybrid is a Julia package for hybrid machine learning models, combining neural networks and traditional statistical methods. It provides tools for data preprocessing, model training, and evaluation, making it easier to build and deploy hybrid models.

The hybrid model combines a neural network h(x; θ), with inputs x and learnable parameters θ, together with a mechanistic model M(·, z; ϕ) driven by forcings z and parameterized by ϕ, where ϕ may be known, learned from data, or fixed.

Documentation https://earthyscience.github.io/EasyHybrid.jl/

EasyHybrid.DataConfig Type

Configuration for data preparation and loading.

Controls array types, observation shuffling, data splitting, cross-validation, and sequence construction for time-series training.

EasyHybrid.HybridParams Type

julia

HybridParams{M<:Function}

A little parametric stub for “the params of function M.” All of your function‐based models become HybridParams{typeof(f)}.

EasyHybrid.InputBatchNorm Type

julia

InputBatchNorm(chs; kwargs...)

A wrapper around BatchNorm that handles 3D sequence input (features, timesteps, batch).

Lux's BatchNorm expects channels in the penultimate dimension, which works for 2D input (features, batch) but fails for 3D input (features, timesteps, batch) where features are in dim 1. This wrapper reshapes 3D input to 2D (features, timesteps * batch) before normalization, then reshapes back. For 2D input, delegates directly to BatchNorm.

EasyHybrid.LinearHM Type

julia

LinearHM(NN, predictors, forcing, targets, β)

A linear hybrid model with a neural network NN, predictors, forcing and targets terms.

EasyHybrid.LinearHM Method

julia

LinearHM(NN, predictors, forcing, β)(ds_k)

Model definition ŷ = α x + β

Apply the linear hybrid model to the input data ds_k (a KeyedArray with proper dimensions). The model uses the neural network NN to compute new α's based on the predictors and then computes ŷ using the forcing term x.

Returns:

A tuple containing the predicted values ŷ and a named tuple with the computed values of α and the state st.

Example:

julia

using Lux
using EasyHybrid
using Random
using AxisKeys

ds_k = KeyedArray(rand(Float32, 3,4); data=[:a, :b, :c], sample=1:4)
m = Lux.Chain(Dense(2, 5), Dense(5, 1))
# Instantiate the model
# Note: The model is initialized with a neural network and the predictors and forcing terms
lh_model = LinearHM(m, (:a, :b), (:c,), 1.5f0)
rng = Random.default_rng()
Random.seed!(rng, 0)
ps, st = LuxCore.setup(rng, lh_model)
# Apply the model to the data
ŷ, αst = LuxCore.apply(lh_model, ds_k, ps, st)

EasyHybrid.LoggingLoss Type

julia

LoggingLoss

A structure to define a logging loss function for hybrid models.

Arguments

loss_types: A vector of loss specifications (Symbol, Function or Tuple)
- Symbol: predefined loss, e.g. :mse
- Function: custom loss function, e.g. custom_loss
- Tuple: function with args/kwargs:
  - (f, args): positional args, e.g. (weighted_loss, (0.5,))
  - (f, kwargs): keyword args, e.g. (scaled_loss, (scale=2.0,))
  - (f, args, kwargs): both, e.g. (complex_loss, (0.5,), (scale=2.0,))
training_loss: The loss specification to use during training (same format as above)
extra_loss: Optional function (ŷ; kwargs...) -> scalar to add to training loss (default: nothing)
agg: Function to aggregate losses across targets, e.g. sum or mean
train_mode: If true, uses training_loss; otherwise uses loss_types.

Examples

julia

# Simple predefined loss
logging = LoggingLoss(
    loss_types=[:mse, :mae],
    training_loss=:mse
)

# Custom loss function
custom_loss(ŷ, y) = mean(abs2, ŷ .- y)
logging = LoggingLoss(
    loss_types=[:mse, custom_loss],
    training_loss=custom_loss
)

# With arguments/kwargs
weighted_loss(ŷ, y, w) = w * mean(abs2, ŷ .- y)
scaled_loss(ŷ, y; scale=1.0) = scale * mean(abs2, ŷ .- y)
logging = LoggingLoss(
    loss_types=[:mse, (weighted_loss, (0.5,)), (scaled_loss, (scale=2.0,))],
    training_loss=(weighted_loss, (0.5,))
)

EasyHybrid.PerTarget Type

julia

PerTarget(losses)

A wrapper to indicate that a tuple of losses should be applied on a per-target basis.

EasyHybrid.RbQ10_2p Type

julia

RbQ10_2p(forcing, targets, Q10)

EasyHybrid.RbQ10_2p Method

julia

RbQ10_2p(NN, predictors, forcing, targets, Q10)(ds_k)

Model definition ŷ = Rb(αᵢ(t)) * Q10^((T(t) - T_ref)/10)

ŷ (respiration rate) is computed as a function of the neural network output Rb(αᵢ(t)) and the temperature T(t) adjusted by the reference temperature T_ref (default 15°C) using the Q10 temperature sensitivity factor. ````

EasyHybrid.RecurrenceOutputDense Type

julia

RecurrenceOutputDense(in_dims => out_dims, [activation])

A layer that wraps a Dense layer to handle sequence outputs from Recurrence layers.

When a Recurrence layer has return_sequence=true, it outputs a tuple/vector of arrays (one per timestep). This layer broadcasts the Dense operation over each timestep and reshapes the result to (features, timesteps, batch) format.

Arguments

in_dims::Int: Input dimension (should match Recurrence output dimension)
out_dims::Int: Output dimension
activation: Activation function (default: identity)

Example

julia

# Instead of manually creating:
broadcast_layer = @compact(; layer = Dense(15 => 15)) do x
    y = map(layer, x)
    @return permutedims(stack(y; dims = 3), (1, 3, 2))
end

# Simply use:
Chain(
    Recurrence(LSTMCell(15 => 15), return_sequence = true),
    RecurrenceOutputDense(15 => 15)
)

EasyHybrid.RespirationRbQ10 Type

julia

RespirationRbQ10(NN, predictors, forcing, targets, Q10)

A linear hybrid model with a neural network NN, predictors, targets and forcing terms.

EasyHybrid.RespirationRbQ10 Method

julia

RespirationRbQ10(NN, predictors, forcing, targets, Q10)(ds_k)

Model definition ŷ = Rb(αᵢ(t)) * Q10^((T(t) - T_ref)/10)

ŷ (respiration rate) is computed as a function of the neural network output Rb(αᵢ(t)) and the temperature T(t) adjusted by the reference temperature T_ref (default 15°C) using the Q10 temperature sensitivity factor. ````

EasyHybrid.Rs_components Type

julia

Rs_components(NN, predictors, forcing, targets, Q10_het, Q10_root, Q10_myc)

A linear hybrid model with a neural network NN, predictors, targets and forcing terms.

EasyHybrid.Rs_components Method

julia

Rs_components(NN, predictors, forcing, targets, Q10)(ds_k)

Model definition ŷ = Rb(αᵢ(t)) * Q10^((T(t) - T_ref)/10)

ŷ (respiration rate) is computed as a function of the neural network output Rb(αᵢ(t)) and the temperature T(t) adjusted by the reference temperature T_ref (default 15°C) using the Q10 temperature sensitivity factor. ````

EasyHybrid.TrainConfig Type

Configuration for training a hybrid model.

Controls all aspects of the training process including optimization, loss computation, data handling, output, and visualization.

EasyHybrid.TrainResults Type

Output of train, containing the full training history, model state, and diagnostics.

EasyHybrid.TrainingPaths Type

Paths to all output files produced during a training run.

EasyHybrid.WrappedTuples Type

julia

WrappedTuples(vec::Vector{<:NamedTuple})

Wraps a vector of named tuples to allow dot-access to each field as a vector.

EasyHybrid._apply_loss Function

julia

_apply_loss(ŷ, y, y_nan, loss_spec)

Helper function to apply the appropriate loss function based on the specification type.

Arguments

ŷ: Predictions for a single target
y: Target values for a single target
y_nan: NaN mask for a single target
loss_spec: Loss specification (Symbol, Function, or Tuple)

Returns

Computed loss value

EasyHybrid._compute_loss Function

julia

_compute_loss(ŷ, y, y_nan, targets, loss_spec, agg::Function)
_compute_loss(ŷ, y, y_nan, targets, loss_types::Vector, agg::Function)

Compute the loss for the given predictions and targets using the specified training loss (or vector of losses) type and aggregation function.

Arguments:

ŷ: Predicted values.
y: Target values.
y_nan: Mask for NaN values.
targets: The targets for which the loss is computed.
loss_spec: The loss type to use during training, e.g., :mse.
loss_types::Vector: A vector of loss types to compute, e.g., [:mse, :mae].
agg::Function: The aggregation function to apply to the computed losses, e.g., sum or mean.

Returns a single loss value if loss_spec is provided, or a NamedTuple of losses for each type in loss_types.

EasyHybrid._get_target_nan Function

julia

_get_target_nan(y_nan, target)

Helper function to extract target-specific values from y_nan.

EasyHybrid._get_target_y Function

julia

_get_target_y(y, target)

Helper function to extract target-specific values from y, handling cases where y can be a tuple of (y_obs, y_sigma).

EasyHybrid.build_parameter_matrix Method

julia

build_parameter_matrix(parameter_defaults_and_bounds::NamedTuple)

Build a ComponentArray matrix from a NamedTuple containing parameter defaults and bounds.

This function converts a NamedTuple where each value is a tuple of (default, lower, upper) bounds into a ComponentArray with named axes for easy parameter management in hybrid models.

Arguments

parameter_defaults_and_bounds::NamedTuple: A NamedTuple where each key is a parameter name and each value is a tuple of (default, lower, upper) for that parameter.

Returns

ComponentArray: A 2D ComponentArray with:
- Row axis: Parameter names (from the NamedTuple keys)
- Column axis: Bound types (:default, :lower, :upper)
- Data: The parameter values organized in a matrix format

Example

julia

# Define parameter defaults and bounds
parameter_defaults_and_bounds = (
    θ_s = (0.464f0, 0.302f0, 0.700f0),     # Saturated water content [cm³/cm³]
    h_r = (1500.0f0, 1500.0f0, 1500.0f0),  # Pressure head at residual water content [cm]
    α   = (log(0.103f0), log(0.01f0), log(7.874f0)),  # Shape parameter [cm⁻¹]
    n   = (log(3.163f0 - 1), log(1.100f0 - 1), log(20.000f0 - 1)),  # Shape parameter [-]
)

# Build the ComponentArray
parameter_matrix = build_parameter_matrix(parameter_defaults_and_bounds)

# Access specific parameter bounds
parameter_matrix.θ_s.default  # Get default value for θ_s
parameter_matrix[:, :lower]   # Get all lower bounds
parameter_matrix[:, :upper]   # Get all upper bounds

Notes

The function expects each value in the NamedTuple to be a tuple with exactly 3 elements
The order of bounds is always (default, lower, upper)
The resulting ComponentArray can be used for parameter optimization and constraint handling

EasyHybrid.build_parameters Method

julia

build_parameters(parameters::NamedTuple, f::DataType) -> AbstractHybridModel

Constructs a parameter container from a named tuple of parameter bounds and wraps it in a user-defined subtype of AbstractHybridModel.

Arguments

parameters::NamedTuple: Named tuple where each entry is a tuple of (default, lower, upper) bounds for a parameter.
f::DataType: A constructor for a subtype of AbstractHybridModel that takes a ParameterContainer as its argument.

Returns

An instance of the user-defined AbstractHybridModel subtype containing the parameter container.

EasyHybrid.compute_loss Method

julia

compute_loss(HM, x, (y_t, y_nan), ps, st, logging::LoggingLoss)

Main loss function for hybrid models that handles both training and evaluation modes.

Arguments

HM: The hybrid model (AbstractLuxContainerLayer or specific model type)
x: Input data for the model
(y_t, y_nan): Tuple containing target values and NaN mask functions/arrays
ps: Model parameters
st: Model state
logging: LoggingLoss configuration

Returns

In training mode (logging.train_mode = true):
- (loss_value, st): Single loss value and updated state
In evaluation mode (logging.train_mode = false):
- (loss_values, st, ŷ): NamedTuple of losses, state and predictions

EasyHybrid.constructNNModel Method

julia

constructNNModel(predictors, targets; hidden_layers, activation, scale_nn_outputs)

Main constructor: hidden_layers can be either • a Vector{Int} of sizes, or • a Chain of hidden-layer Dense blocks.

EasyHybrid.evec Method

evec(nt::NamedTuple)

EasyHybrid.filter_sequences Method

julia

filter_sequences(x, y) -> (x_filtered, y_filtered)

Drop 3rd-dim samples where any predictor is NaN or all targets are NaN.

EasyHybrid.get_loss_value Method

julia

get_loss_value(losses, loss_spec, agg)

Extract loss value from losses based on the loss specification type.

Arguments

losses: NamedTuple containing loss values
loss_spec: Loss specification (Symbol, Function or Tuple)
agg: Aggregation function name as Symbol

Returns

Loss value for the specified loss function

Examples

julia

# Symbol case
val = get_loss_value(losses, :mse, :sum)

# Function case
custom_loss(ŷ, y) = mean(abs2, ŷ .- y)
val = get_loss_value(losses, custom_loss, :mean)

# Tuple case
weighted_loss(ŷ, y, w) = w * mean(abs2, ŷ .- y)
val = get_loss_value(losses, (weighted_loss, (0.5,)), :sum)

EasyHybrid.get_prediction_target_names Method

julia

get_prediction_target_names(hm)

Utility function to extract predictor/forcing and target names from a hybrid model.

Arguments:

hm: The Hybrid Model

Returns a tuple of (predictors_forcing, targets) names.

EasyHybrid.initialize_plotting_observables Method

julia

initialize_plotting_observables(init_ŷ_train, init_ŷ_val, y_train, y_val, l_init_train, l_init_val, training_loss, agg, monitor_names, target_names)

Initialize plotting observables for training visualization if the Makie extension is loaded.

EasyHybrid.load_timeseries_netcdf Method

julia

load_timeseries_netcdf(path::AbstractString; timedim::AbstractString = "time") -> DataFrame

Reads a NetCDF file where all data variables are 1D over the specified timedim and returns a tidy DataFrame with one row per time step.

Only includes variables whose sole dimension is timedim.
Does not attempt to parse or convert time units; all columns are read as-is.

EasyHybrid.loss_fn Function

julia

loss_fn(ŷ, y, y_nan, loss_type)

Compute the loss for given predictions and targets using various loss specifications.

Arguments

ŷ: Predicted values
y: Target values
y_nan: Mask for NaN values
loss_type: One of the following:
- Val(:rmse): Root Mean Square Error
- Val(:mse): Mean Square Error
- Val(:mae): Mean Absolute Error
- Val(:pearson): Pearson correlation coefficient
- Val(:r2): R-squared
- Val(:pearsonLoss): 1 - Pearson correlation coefficient
- Val(:nseLoss): 1 - NSE
- ::Function: Custom loss function with signature f(ŷ, y)
- ::Tuple{Function, Tuple}: Custom loss with args f(ŷ, y, args...)
- ::Tuple{Function, NamedTuple}: Custom loss with kwargs f(ŷ, y; kwargs...)
- ::Tuple{Function, Tuple, NamedTuple}: Custom loss with both f(ŷ, y, args...; kwargs...)

Examples

julia

# Predefined loss
loss = loss_fn(ŷ, y, y_nan, Val(:mse))

# Custom loss function
custom_loss(ŷ, y) = mean(abs2, ŷ .- y)
loss = loss_fn(ŷ, y, y_nan, custom_loss)

# With positional arguments
weighted_loss(ŷ, y, w) = w * mean(abs2, ŷ .- y)
loss = loss_fn(ŷ, y, y_nan, (weighted_loss, (0.5,)))

# With keyword arguments
scaled_loss(ŷ, y; scale=1.0) = scale * mean(abs2, ŷ .- y)
loss = loss_fn(ŷ, y, y_nan, (scaled_loss, (scale=2.0,)))

# With both args and kwargs
complex_loss(ŷ, y, w; scale=1.0) = scale * w * mean(abs2, ŷ .- y)
loss = loss_fn(ŷ, y, y_nan, (complex_loss, (0.5,), (scale=2.0,)))

You can define additional predefined loss functions by adding more methods:

julia

import EasyHybrid: loss_fn
function EasyHybrid.loss_fn(ŷ, y, y_nan, ::Val{:nse})
    return 1 - sum((ŷ[y_nan] .- y[y_nan]).^2) / sum((y[y_nan] .- mean(y[y_nan])).^2)
end

EasyHybrid.make_folds Method

julia

make_folds(df::DataFrame; k::Int=5, shuffle=true) -> Vector{Int}

Assigns each observation in the DataFrame df to one of k folds for cross-validation.

Arguments

df::DataFrame: The input DataFrame whose rows are to be split into folds.
k::Int=5: Number of folds to create.
shuffle=true: Whether to shuffle the data before assigning folds.

Returns

folds::Vector{Int}: A vector of length nrow(df) where each entry is an integer in 1:k indicating the fold assignment for that observation.

EasyHybrid.prepare_data Function

julia

prepare_data(hm, data::DataFrame; array_type=:KeyedArray, drop_missing_rows=true)
prepare_data(hm, data::KeyedArray)
prepare_data(hm, data::AbstractDimArray)
prepare_data(hm, data::Tuple)

Prepare data for training by extracting predictor/forcing and target variables based on the hybrid model's configuration.

Arguments:

hm: The Hybrid Model
data: The input data, which can be a DataFrame, KeyedArray, or DimensionalData array.
array_type: (DataFrame only) Output array type: :KeyedArray (default) or :DimArray.
drop_missing_rows: (DataFrame only) If true (default), drop rows where any predictor is NaN or all targets are NaN.

Returns:

If data is a DataFrame: a tuple of (predictors_forcing, targets) as KeyedArrays or DimArrays depending on array_type.
If data is a KeyedArray: a tuple of (predictors_forcing, targets) as KeyedArrays.
If data is an AbstractDimArray: a tuple of (predictors_forcing, targets) as DimArrays.
If data is already a Tuple, it is returned as-is.

EasyHybrid.prepare_hidden_chain Method

julia

prepare_hidden_chain(hidden_layers, in_dim, out_dim; activation, input_batchnorm=false)

Construct a neural network Chain for use in NN models.

Arguments

hidden_layers::Union{Vector{Int}, Chain}:
- If a Vector{Int}, specifies the sizes of each hidden layer. For example, [32, 16] creates two hidden layers with 32 and 16 units, respectively.
- If a Chain, the user provides a pre-built chain of hidden layers (excluding input/output layers). If the chain ends with a Recurrence layer, a RecurrenceOutputDense layer is automatically added to handle the sequence output format.
in_dim::Int: Number of input features (input dimension).
out_dim::Int: Number of output features (output dimension).
activation: Activation function to use in hidden layers (default: tanh).
input_batchnorm::Bool: If true, applies a BatchNorm layer to the input (default: false).

Returns

A Chain object representing the full neural network, with the following structure:
- Optional input batch normalization (if input_batchnorm=true)
- Input layer: Dense(in_dim, h₁, activation) where h₁ is the first hidden size
- Hidden layers: either user-supplied Chain or constructed from hidden_layers
- If last hidden layer is a Recurrence, a RecurrenceOutputDense is added to handle sequence output
- Output layer: Dense(hₖ, out_dim) where hₖ is the last hidden size

where h₁ is the first hidden size and hₖ the last.

Example with Recurrence (LSTM)

julia

# User only needs to define:
NN_Memory = Chain(
    Recurrence(LSTMCell(15 => 15), return_sequence = true),
)

# The function automatically adds the RecurrenceOutputDense layer to handle sequence output
model = constructHybridModel(..., hidden_layers = NN_Memory, ...)

EasyHybrid.scale_single_param Method

julia

scale_single_param(name, raw_val, parameters)

Scale a single parameter using the sigmoid scaling function.

EasyHybrid.scale_single_param_minmax Method

julia

scale_single_param_minmax(name, hm::AbstractHybridModel)

Scale a single parameter using the minmax scaling function.

EasyHybrid.select_cols Method

select_cols(df, predictors, x)

EasyHybrid.select_cols Method

select_cols(df::KeyedArray, predictors, x)

EasyHybrid.select_predictors Method

select_predictors(df, predictors)

EasyHybrid.select_predictors Method

select_predictors(dk::KeyedArray, predictors)

EasyHybrid.select_variable Method

select_variable(df::KeyedArray, x)

EasyHybrid.split_data Function

julia

split_data(data, hybridModel; split_by_id=nothing, shuffleobs=false, split_data_at=0.8, kwargs...)
split_data(data::Union{DataFrame, KeyedArray}, hybridModel; split_by_id=nothing, shuffleobs=false, split_data_at=0.8, folds=nothing, val_fold=nothing, kwargs...)
split_data(data::AbstractDimArray, hybridModel; split_by_id=nothing, shuffleobs=false, split_data_at=0.8, kwargs...)
split_data(data::Tuple, hybridModel; split_by_id=nothing, shuffleobs=false, split_data_at=0.8, kwargs...)
split_data(data::Tuple{Tuple, Tuple}, hybridModel; kwargs...)

Split data into training and validation sets, either randomly, by grouping by ID, or using external fold assignments.

Arguments:

data: The data to split, which can be a DataFrame, KeyedArray, AbstractDimArray, or Tuple
hybridModel: The hybrid model object used for data preparation
split_by_id=nothing: Either nothing for random splitting, a Symbol for column-based splitting, or an AbstractVector for custom ID-based splitting
shuffleobs=false: Whether to shuffle observations during splitting
split_data_at=0.8: Ratio of data to use for training
folds: Vector or column name of fold assignments (1..k), one per sample/column for k-fold cross-validation
val_fold: The validation fold to use when folds is provided
sequence_kwargs=nothing: NamedTuple of keyword arguments forwarded to split_into_sequences (e.g. (; input_window=10, output_window=1, output_shift=1, lead_time=2)). When set, data is windowed into 3D sequences before splitting.

Behavior:

For DataFrame/KeyedArray: Supports random splitting, ID-based splitting, and external fold assignments
For AbstractDimArray/Tuple: Random splitting only after data preparation
For pre-split Tuple{Tuple, Tuple}: Returns input unchanged

Returns:

((x_train, y_train), (x_val, y_val)): Tuple containing training and validation data pairs

EasyHybrid.split_data Method

split_data(df::DataFrame, target, xvars, seqID; f=0.8, batchsize=32, shuffle=true, partial=true)

EasyHybrid.split_data Method

split_data(df::DataFrame, target, xvars; f=0.8, batchsize=32, shuffle=true, partial=true)

EasyHybrid.split_into_sequences Method

julia

split_into_sequences(x, y; input_window=5, output_window=1, output_shift=1, lead_time=1)

Slide a (input_window + lead_time) window over 2D (feature, time) arrays to produce 3D (feature, time, batch) tensors for sequence-to-sequence training.

Arguments:

x: 2D input array (feature, time).
y: 2D target array (target, time).
input_window: number of input time steps per sample.
output_window: number of target time steps per sample.
output_shift: stride between consecutive samples.
lead_time: gap between end of input window and end of output window.

Returns:

(X, Y) as KeyedArrays or DimArrays matching the input type.

EasyHybrid.toDataFrame Method

julia

toDataFrame(arr, target_names)

Extract specific target variables from a labeled array into a DataFrame with _pred suffix.

Arguments

arr: A labeled array or NamedTuple-like object with property access
target_names: Vector of target variable names to extract

Returns

DataFrame with columns named <target>_pred for each target

EasyHybrid.toDataFrame Method

julia

toDataFrame(arr::Union{KeyedArray{T, 2}, AbstractDimArray{T, 2}}, cols_dim=:variable, index_dim=:batch_size; index_col=:index)

Convert a 2D labeled array (KeyedArray or DimArray) to a DataFrame.

Arguments

arr: The 2D labeled array to convert
cols_dim: Dimension name to use as DataFrame columns (default: :variable)
index_dim: Dimension name to use as DataFrame row index (default: :batch_size)
index_col: Name for the index column in the result (default: :index)

Returns

DataFrame with columns from cols_dim keys and an index column from index_dim keys

EasyHybrid.toDataFrame Method

julia

toDataFrame(arr::AbstractLabeledArray{T, 3}, cols_dim=:variable, index_dim=:batch_size; slice_dim=:time, index_col=:index)

Convert a 3D labeled array (KeyedArray or DimArray) to a Dict of DataFrames, one per slice.

Arguments

arr: The 3D labeled array to convert
cols_dim: Dimension name to use as DataFrame columns (default: :variable)
index_dim: Dimension name to use as DataFrame row index (default: :batch_size)
slice_dim: Dimension name to slice along (default: :time)
index_col: Name for the index column in each result DataFrame (default: :index)

Returns

Dict{Any, DataFrame} mapping slice keys to DataFrames

EasyHybrid.toNamedTuple Method

toNamedTuple(ka::KeyedArray, variable::Symbol) Extract a single variable from a KeyedArray and return it as a vector.

Arguments:

ka: The KeyedArray or DimArray to unpack
variable: Symbol representing the variable to extract

Returns:

Vector containing the variable data

Example:

julia

# Extract just SW_IN from an array
sw_in = toNamedTuple(ds, :SW_IN)

EasyHybrid.toNamedTuple Method

julia

toNamedTuple(ka::Union{KeyedArray, AbstractDimArray}, variables::Vector{Symbol})

Extract specified variables from a KeyedArray or DimArray and return them as a NamedTuple of vectors.

Arguments:

ka: The KeyedArray or DimArray to unpack
variables: Vector of symbols representing the variables to extract

Returns:

NamedTuple with variable names as keys and vectors as values

Example:

julia

# Extract SW_IN and TA from an array
data = toNamedTuple(ds, [:SW_IN, :TA])
sw_in = data.SW_IN
ta = data.TA

EasyHybrid.toNamedTuple Method

toNamedTuple(ka::KeyedArray) Extract all variables from a KeyedArray and return them as a NamedTuple of vectors.

Arguments:

ka: The KeyedArray to unpack

Returns:

NamedTuple with all variable names as keys and vectors as values

Example:

julia

# Extract all variables from an array
data = toNamedTuple(ds)
# Access individual variables
sw_in = data.SW_IN
ta = data.TA
nee = data.NEE

EasyHybrid.to_dimArray Method

to_dimArray(df::DataFrame)

EasyHybrid.to_keyedArray Function

tokeyedArray(dfg::Union{Vector,GroupedDataFrame{DataFrame}}, vars=All())

EasyHybrid.to_keyedArray Method

tokeyedArray(df::DataFrame)

EasyHybrid.train Method

julia

train(model, data; train_cfg::TrainConfig = TrainConfig(), data_cfg::DataConfig = DataConfig())

Train a hybrid model using the provided data.

Returns nothing if data preparation fails (zero-size dimension in training or validation data).

Arguments

model: The hybrid model to train.
data: Training data, a single DimArray, a single DataFrame, a single KeyedArray, or a tuple of those.

Keyword Arguments

train_cfg: Training configuration. See TrainConfig for all options.
data_cfg: Data preparation configuration. See DataConfig for all options.

Returns

A TrainResults with the following fields:

train_losses: Per-epoch training losses.
val_losses: Per-epoch validation losses.
snapshots: Model parameter snapshots taken during training.
train_obs_pred: Observed vs. predicted values on the training set.
val_obs_pred: Observed vs. predicted values on the validation set.
train_diffs: Additional diagnostic variables computed on the training set.
val_diffs: Additional diagnostic variables computed on the validation set.
ps: Final (or best) model parameters.
st: Final (or best) model state.
best_epoch: Epoch at which the best validation loss was achieved.
best_loss: Best validation loss recorded during training.

Example

julia

cfg = TrainConfig(nepochs=100, batchsize=32)
result = train(myModel, myData; train_cfg=cfg)

EasyHybrid.@hybrid Macro

julia

@hybrid ModelName α β γ

Macro to define hybrid model structs with arbitrary numbers of physical parameters.

This defines a struct with:

Default fields: NN (neural network), predictors, forcing, targets.
Additional physical parameters, i.e., α β γ.

Examples

julia

@hybrid MyModel α β γ
@hybrid FluidModel (:viscosity, :density)
@hybrid SimpleModel :a :b