ReducedComplexityModeling

ReducedComplexityModeling

Reduced models of the solutions of geometric integrators: the snapshot data of a solution or an ensemble of solutions, reduced bases (ReducedBasisModel, POD), parameters and parameter spaces with their samplers, the fit of a model's parameters to data (TrainingProblem, train), and the loaders that draw batches of data for a neural network (DataLoader, Batch).

Batch(batch_size, seq_length)

Make an instance of Batch for a specific batch size and a sequence length.

This is used to train neural networks of TransformerIntegrator type.

Optionally the prediction window can also be specified by calling:

using ReducedComplexityModeling

batch_size = 2
seq_length = 3
prediction_window = 2

Batch(batch_size, seq_length, prediction_window)

# output

Batch{:Transformer}(2, 3, 2)

Note that here the batch is of type :Transformer.

Batch

Batch is a struct whose functor acts on an instance of DataLoader to produce a sequence of training samples for training for one epoch.

See Batch(::Int) and Batch(::Int, ::Int, ::Int) for the different constructors.

The functor

An instance of Batch can be called on an instance of DataLoader to produce a sequence of samples that contain all the input data, i.e. for training for one epoch.

The output of applying batch::Batch to dl::DataLoader is a vector of minibatches, each a vector of tuples of two integers: the first is the time index and the second one is the parameter index.

The minibatches of an epoch

The functor shuffles the start positions and the parameter indices separately, takes their product in column order and cuts this list into minibatches of the batch size. Every pair of a time index and a parameter index appears once, and the start positions of every parameter come in the same order. So an epoch runs through the parameters one after the other, and a minibatch is not a uniform sample of the pairs.

With n start positions per parameter and P parameters, a batch size b ≤ n leaves at most P - 1 minibatches with more than one parameter. The mean of a per-sample gradient over a minibatch therefore has a variance between the parameters that does not fall with b.

Where one axis has length one, the minibatches are a uniform partition of the indices. This holds for a matrix or a vector with autoencoder = true and for one trajectory. A tensor with autoencoder = true keeps both of its axes.

Examples

Consider the following example for drawing batches of size 2 for an instance of DataLoader constructed with a vector:

using ReducedComplexityModeling
import Random

rng = Random.TaskLocalRNG()
Random.seed!(rng, 123)

dl = DataLoader(rand(rng, 5); autoencoder = true)
batch = Batch(2)

batches = batch(dl)

@assert length(batches) == 3
@assert length.(batches) == [2, 2, 1]
@assert sort(collect(union(batches...))) == [(1, i) for i in 1:5]

true

# output

[ Info: You have provided a matrix as input. The axes will be interpreted as (i) system dimension and (ii) number of parameters.
true

Here the first index is always 1 (the time dimension). We get a total number of 3 batches. The last batch is only of size 1 because we sample without replacement. Also see the docstring for DataLoader(::AbstractVector).

A tensor with 6 time steps and 3 parameters has 5 start positions per parameter for a Batch with a sequence length and a prediction window of 1:

using ReducedComplexityModeling

dl = DataLoader(rand(1, 6, 3); suppress_info = true)
batches = Batch(2, 1, 1)(dl)

@assert length(batches) == 8
@assert count(minibatch -> length(unique(last.(minibatch))) > 1, batches) == 1

true

# output

true

Only one of the 8 minibatches holds two parameters, and each of the others holds one.

Batch(batch_size)

Make an instance of Batch for a specific batch size.

This is, among others, used to train neural networks of NeuralNetworkIntegrator type (as opposed to TransformerIntegrator).

CartesianParameterSampler()

Draws every combination of the samples stored in the parameters: the Cartesian product of their sample vectors, with the first parameter varying fastest. Each parameter must have samples (see hassamples); a parameter without them throws an AssertionError. The samples keep the element type of each parameter.

DataLoader(data)

Make an instance based on a data set.

This is designed to make training convenient.

Fields of DataLoader

The fields of the DataLoader struct are the following:

• input: The input data with axes (i) system dimension, (ii) number of time steps and (iii) number of parameters.
• output: The tensor that contains the output (supervised learning) - this may be of type Nothing if the constructor is only called with one tensor (unsupervised learning).
• input_dim: The dimension of the system, i.e. what is taken as input by a regular neural network.
• input_time_steps: The length of the entire time series (length of the second axis).
• n_params: The number of parameters that are present in the data set (length of third axis)
• output_dim: The dimension of the output tensor (first axis). If output is of type Nothing, then this is also of type Nothing.
• output_time_steps: The size of the second axis of the output tensor. If output is of type Nothing, then this is also of type Nothing.

Implementation

Even though DataLoader can be called with inputs of various forms, internally it always stores tensors with three axes.

using ReducedComplexityModeling

data = [1 2 3; 4 5 6]
dl = DataLoader(data)
dl.input

# output

[ Info: You have provided a matrix as input. The axes will be interpreted as (i) system dimension and (ii) number of time steps.
2×3×1 Array{Int64, 3}:
[:, :, 1] =
 1  2  3
 4  5  6

DataLoader(data::AbstractVector)

Make an instance of DataLoader based on a vector.

Extended help

If the input to DataLoader is a vector, it is assumed that this vector represents one-dimensional time-series data and is therefore processed as:

    DataLoader(data::AbstractVector; autoencoder=false) = DataLoader(reshape(data, 1, length(data)); autoencoder = autoencoder)

DataLoader(data::NamedTuple{(:q, :p), Tuple{AT, AT}})

Make an instance of DataLoader based on $(q, p)$ data.

Implementation

In this case the field input_dim of DataLoader is interpreted as the sum of the $q$- and $p$-dimensions, i.e. if $q$ and $p$ both evolve on $\mathbb{R}^n$, then input_dim is $2n$.

Apart from this the input is treated similarly as if it were an Array, i.e. everything is converted to tensors internally. See e.g. DataLoader{::AbstractArray{<:Number, 3}}.

DataLoader(data::AbstractArray{<:Number, 3})

Make an instance of DataLoader for data that are in tensor format.

Arguments

There are two optional keyword arguments:

• autoencoder = false and
• suppress_info = false.

By default the data are stored as TimeSeries type. If you want to train an AutoEncoder with your data call:

    DataLoader(data; autoencoder = true)

The default is equivalent to autoencoder = false.

By default we have:

using ReducedComplexityModeling

data = [ 1;  2;  3;;
         4;  5;  6;;;
         7;  8;  9;;
        10; 11; 12]

DataLoader(data)

# output

┌ Info: You have provided a tensor with three axes as input. They will be interpreted as
└  (i) system dimension, (ii) number of time steps and (iii) number of params.
DataLoader{Int64, Array{Int64, 3}, Nothing, :TimeSeries, Nothing}([1 4; 2 5; 3 6;;; 7 10; 8 11; 9 12], nothing, 3, 2, 2, nothing, nothing)

But if we write

using ReducedComplexityModeling

data = [ 1;  2;  3;;
         4;  5;  6;;;
         7;  8;  9;;
        10; 11; 12]

DataLoader(data; suppress_info = true)

# output

DataLoader{Int64, Array{Int64, 3}, Nothing, :TimeSeries, Nothing}([1 4; 2 5; 3 6;;; 7 10; 8 11; 9 12], nothing, 3, 2, 2, nothing, nothing)

the @info statement is not printed.

DataLoader(data::AbstractMatrix)

Make an instance of DataLoader based on a matrix.

Arguments

See DataLoader(::AbstractArray{<:Number, 3}) for details.

Implementation

The second axis of the matrix is the number of time steps, and internally the data are reshaped to a tensor of shape (size(data)..., 1): one trajectory of one parameter. With autoencoder = true the matrix is a set of samples instead, so that axis is the parameter index and the time axis has length one.

DataLoader(dl, backend)

Make a new instance of DataLoader based on an existing instance of DataLoader for a new backend.

This allocates new memory of the same size as is used for the original dl and then copies the data.

By default the data type remains unchanged, i.e. eltype(DataLoader(dl, backend)) == eltype(dl) is true.

If you want to change the data type write e.g.

    DataLoader(dl, backend, Float32)

Arguments

There is an optional keyword argument

• autoencoder = nothing.

By default this inherits the autoencoder property from dl.

See the docstring for DataLoader(data::AbstractArray{<:Number, 3}).

DataLoader(ensemble_solution)

Make an instance of DataLoader for an EnsembleSolution.

EnsembleSolutions are imported from the package GeometricSolutions.jl.

Arguments

This functor for DataLoader also has the keyword arguments

• autoencoder = false and
• suppress_info = false.

See the docstring for DataLoader(::AbstractArray{<:Number, 3}).

Implementation

Internally this stores the data as a tensor where the third axis has length equal to the number of solutions in the ensemble.

DataLoader(solution)

Make an instance of DataLoader for a GeometricSolution.

GeometricSolutions are imported from the package GeometricSolutions.jl.

Arguments

This functor for DataLoader also has the keyword arguments

• autoencoder = false and
• suppress_info = false.

See the docstring for DataLoader(::AbstractArray{<:Number, 3}).

Implementation

Internally this stores the data as a tensor where the third axis has length 1.

Dependent()

The layout of the snapshot of a GeometricIntegratorEnsembleData in which the solutions form one snapshot matrix: one row per component of the state q, and the entries of q of the first solution, then those of the second, and so on, as its columns. The solutions may have different numbers of entries in q, but all must have the same number of components of q; otherwise the constructor throws an ArgumentError.

GeometricIntegratorData(solution::GeometricSolution)

The solution of a geometric integrator and its snapshot matrix: the field snapshot is a matrix with one row per component of the state q and one column per entry of solution.q, the initial state included.

GeometricIntegratorEnsembleData(solutions::AbstractVector{<:GeometricSolution}, deps)

The solutions of an ensemble of geometric integrations and their snapshot, in the layout that deps selects: Dependent or Independent. The element type of the snapshot is the promotion of the element types of the states q of all solutions.

Independent()

The layout of the snapshot of a GeometricIntegratorEnsembleData in which each solution keeps its own snapshot, along a third dimension: an array of size ny × nx × ns, for ns solutions with nx entries of a state q with ny components. Its slice [:, :, i] is the snapshot matrix of solution i, in the layout of GeometricIntegratorData. All solutions must have the same number of entries in q and the same number of components of q; otherwise the constructor throws an ArgumentError.

IntrinsicSpace

The space in which the structure of the system is manifest.

IntrinsicSpace marks structure, not size. A system may be canonically Hamiltonian while its observables are not canonically conjugate variables, and the intrinsic coordinates are then reached by a coordinate transformation or a learned map at the same dimension as the observables; dimension reduction is only one way to reach them, so a reduced basis and a canonicalising transformation are both maps ObservableSpace → IntrinsicSpace.

The map itself is not part of this type. It stays with the model that produced the data, for example the project and lift of a reduced basis or the transformation of a symbolic-regression method, so that each map has one owner. A package adds a subtype of IntrinsicSpace to name its own kind of map.

ObservableSpace

The space of the recorded variables: what a measurement, a simulation or a file produced.

POD()

Proper orthogonal decomposition: reduce!(model, k, POD()) takes the first k left singular vectors of the snapshot matrix as the reduced basis.

Parameter(name, minimum, maximum)
Parameter(name, minimum, maximum, samples)
Parameter(name, minimum, maximum, n::Int)
Parameter(name, samples::AbstractVector)

A named parameter of a system: the bounds minimum ≤ maximum and, optionally, a vector of samples between them. In every form, name is a Symbol or a string, which is converted to a Symbol.

The bounds have one type DT, and the samples have element type DT. The samples are stored sorted and without duplicates. Without samples, samples is nothing. With a count n, the samples are the points of LinRange(minimum, maximum, n), without duplicates. A point that rounds outside the bounds, by a few ULPs, is moved onto the nearest bound. Equal bounds and a positive count give the one sample minimum. This works for bounds of a type that LinRange keeps, such as a float or a Rational; unequal integer bounds with a count throw a MethodError, because their LinRange has a float element type. With only a vector of samples, the bounds are its smallest and largest entry.

minimum > maximum, or a sample outside [minimum, maximum], throws an AssertionError.

ParameterSampler

The supertype of the samplers that sample draws the points of a parameter space with: CartesianParameterSampler, RandomParameterSampler and QuasiRandomParameterSampler.

ParameterSpace collects all parameters of a system as well as samples in the parameter space.

ParametricDataLoader(data, parameters)
ParametricDataLoader(ensemble_solution)

Make a loader of trajectories of a system together with the parameters $\mu$ of the system that each trajectory belongs to.

data is a 2d × nt × m array, or a (q, p) named tuple of two d × nt × m arrays of one array type: m trajectories of nt time steps, with q above p where they are stacked. parameters is a vector of m named tuples, the $\mu$ of each trajectory. Every $\mu$ has the keys of the first, in the same order and at every level, and the same sizes of its leaves. The data and every $\mu$ have one element type. An EnsembleSolution of a problem with (q, p) states gives its q and p and the parameters of its problem.

The batch

A Batch(batch_size) draws, in one epoch, each pair of a start position and a trajectory once, in minibatches of batch_size; the last minibatch is shorter. A drawn minibatch is the triple (input, output, μs): the state at the start position, the state one time step later, and the $\mu$ of each sample. GeometricMachineLearning's train! method for a ParametricDataLoader trains on ParametricLoss() when no loss is given, so the network is applied with apply_parametric, one time step per sample. A Batch with a sequence length or a prediction window other than one is an ArgumentError.

A problem without system parameters is trained with a DataLoader.

QuasiRandomParameterSampler(n)

Draws the first n points of the Halton sequence, scaled to the box spanned by the minimum and maximum of each parameter. The j-th parameter uses the j-th prime as its base, and the sequence starts at index one, so the corner minimum is not a sample. The samples stored in a Parameter are ignored. The samples of a parameter with element type DT have type float(DT), so an integer parameter gets Float64 samples.

The points are deterministic and fill the box more evenly than random draws. The Halton sequence degrades for many parameters, as the bases grow and neighbouring dimensions correlate.

RandomParameterSampler(n, rng = Random.default_rng())

Draws n points uniformly at random from the box spanned by the minimum and maximum of each parameter. The samples stored in a Parameter are ignored. The samples of a parameter with element type DT have type float(DT), so an integer parameter gets Float64 samples.

The draws come from rng, parameter by parameter in the order the parameters are passed, so a sampler built with a seeded generator, e.g. RandomParameterSampler(n, Random.Xoshiro(42)), reproduces the same samples exactly. The generator advances with each call to sample.

ReducedBasisModel(alg, snapshot)

A reduced-basis model of the snapshot matrix snapshot, whose columns are the snapshots, with the reduction scheme alg, such as POD. reduce! computes the reduced basis and stores it in the field reduced_basis, which is undefined until then.

TrainingData{SpaceType, DataType, SystemType}(time, data[, problem])
TrainingData{SpaceType, DataType, SystemType}(solution::EnsembleSolution)

Training data on three orthogonal axes.

The type parameters are

• SpaceType <: Union{ObservableSpace, IntrinsicSpace}: ObservableSpace for recorded data or IntrinsicSpace for data in which the structure is manifest. A method that expects one must not silently accept the other, so the space is a type parameter and not a field;
• DataType <: GeometricBase.AbstractDataType: what was recorded, one of ObservableData, StateData, VectorFieldData and TangentVectorData of GeometricBase;
• SystemType <: GeometricBase.AbstractSystem: the structure of the system, one of the concrete systems of GeometricBase, such as CanonicalHamiltonianSystem or RegularLagrangianSystem.

DataType and SystemType are the axes that GeometricBase already carries: the pair dispatches state_symbols, which returns the state variable names, e.g. (:q, :p) for StateData on a CanonicalHamiltonianSystem and (:z,) on a noncanonical one. A TrainingData adds the space axis and the data themselves; it redefines neither of the others.

SystemType is declared by the caller and not derived from a problem. The hashamiltonian/haslagrangian traits of a GeometricEquations problem separate a Hamiltonian problem from a Lagrangian one and nothing finer, and data need not come from a problem at all.

Every constructor gives the same layout: time is a vector and data an array with the time along its last dimension, so time has size(data, ndims(data)) entries, and a vector, a matrix and a 3-array are all accepted. A shape that does not match throws a DimensionMismatch. The first constructor is called with two or three arguments: with two it sets problem to nothing, with three it stores the problem it is given. The second one builds the array from the ensemble: data[:, i, n] holds the variables that state_symbols names, e.g. q then p for StateData on a CanonicalHamiltonianSystem, of member i at the n-th stored time, and time is the 1-based vector of the stored times. It stores solution.problem itself, not a copy, and throws an ArgumentError when state_symbols names no variable or one that the solution does not have.

DataType and SystemType must be concrete; anything else throws an ArgumentError, rather than letting state_symbols construct a system the caller did not mean.

TrainingProblem(data, residual, parameters)

A training problem: the data, the residual to minimise over the parameters, and the samples the fit starts from.

data is a TrainingData, parameters a ParameterSpace, and residual a callable

residual(data::TrainingData, θ::NamedTuple) -> AbstractVector

that returns the residual vector of the model with parameters θ. The parameters come last, as in the vector fields of a GeometricEquations problem. θ has the keys and the order of parameters.parameters; its element type is generic, so the residual sees ForwardDiff.Dual entries while train differentiates it.

_create_group(h5, name)

The group name of h5: the existing one if there is one, else a new group that tracks the creation order of its links, so that keys returns them in the order written.

accuracy(nn, dl)

Compute the accuracy of a neural network classifier.

This is like accuracy(::Chain, ::NetworkParameters, ::DataLoader), but for a NeuralNetwork.

accuracy(model, ps, dl)

Compute the accuracy of a neural network classifier.

This needs an instance of DataLoader that stores the test data. The result is one minus the fraction of errors, where a wrongly labelled sample is one error and a sample that carries no label is half an error. Its type is float(T), for T the element type of the loader's input.

assign_output_estimate(full_output, prediction_window)

Crop the output to get the correct number of output vectors.

The function assign_output_estimate is closely related to the Transformer. It takes the last prediction_window columns of the output and uses them for the prediction.

i.e.

\[\mathbb{R}^{N\times{}T}\to\mathbb{R}^{N\times\mathtt{pw}}, \begin{bmatrix} z^{(1)}_1 & \cdots & z^{(T)}_1 \\ \cdots & \cdots & \cdots \\ z^{(1)}_n & \cdots & z^{(T)}_n \end{bmatrix} \mapsto \begin{bmatrix} z^{(T - \mathtt{pw} + 1)}_1 & \cdots & z^{(T)}_1 \\ \cdots & \cdots & \cdots \\ z^{(T - \mathtt{pw} + 1)}_n & \cdots & z^{(T)}_n\end{bmatrix}\]

If prediction_window is equal to sequence_length, then this is not needed.

compose(prob::ODEProblem, model::ReducedBasisModel)

The reduced problem of prob in the reduced basis R = model.reduced_basis: an ODEProblem in the reduced coordinates u, with the vector field R' * v(R * u), where v is the vector field of prob, and the initial condition R' * q₀. The time span, the time step and the parameters are those of prob. Call reduce! on model first.

The reduced vector field keeps two buffers of the full state, in the element type of the basis, and does not allocate where u has that element type and the vector field of prob does not allocate. A call with another element type, such as the ForwardDiff.Dual numbers of an implicit integrator's Jacobian, allocates its own arrays. The buffers belong to the reduced problem, so one integrator at a time uses it.

convert_input_and_batch_indices_to_array(dl, batch, batch_indices)

Assign batch data based on (i) input and (ii) batch indices.

Examples

using ReducedComplexityModeling

dl = DataLoader([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9];
    autoencoder = true, suppress_info = true)
batch = Batch(3)
batch_indices = [(1, 1), (1, 3), (1, 5)]

ReducedComplexityModeling.convert_input_and_batch_indices_to_array(dl, batch, batch_indices)

# output

1×1×3 Array{Float64, 3}:
[:, :, 1] =
 0.1

[:, :, 2] =
 0.3

[:, :, 3] =
 0.5

data_tensors_from_geometric_solution(solution)

The data of one GeometricSolution as the tensors a DataLoader takes, with the axes (system dimension, time steps, 1).

A solution with the fields (:t, :q, :p, :q̇, :ṗ) gives the named tuple (q = ..., p = ...), and one with (:t, :q, :v) or (:t, :q, :q̇) gives the tensor of q.

h5load(ParameterSpace, h5::H5DataStore; path = "/")
h5load(Parameter, h5::H5DataStore; path = "/")
h5load(T::Type, fpath::AbstractString, args...; mode = "r", kwargs...)

Read a ParameterSpace or a Parameter from the group path of h5, in the layout that h5save writes. A parameter takes the name of its group. With a file path, open the file fpath in mode and read a T from it.

h5save writes a parameter p to the group of its name, so read it back with path = string(p.name):

julia> p = Parameter(:μ, 0.0, 1.0, 3);

julia> file = joinpath(mktempdir(), "parameter.h5");

julia> h5save(file, p);

julia> h5load(Parameter, file; path = string(p.name)) == p
true

h5save(h5::H5DataStore, ps::ParameterSpace; path = "/")
h5save(h5::H5DataStore, param::Parameter; path = string(param.name))
h5save(fpath::AbstractString, data, args...; mode = "w", kwargs...)

Write ps or param to the group path of h5. A ParameterSpace writes one dataset per parameter to the subgroup samples, and each of its parameters to the subgroup parameters. A Parameter writes the datasets minimum, maximum and, if it has samples, samples. With a file path, open the file fpath in mode and write data to it. h5load reads the data back.

julia> ps = ParameterSpace(Parameter(:μ, 0.0, 1.0, 3), Parameter(:ν, 1.0, 2.0, 2));

julia> file = joinpath(mktempdir(), "parameterspace.h5");

julia> h5save(file, ps);

julia> h5load(ParameterSpace, file) == ps
true

hassamples(p::Parameter)

Whether the parameter p has a non-empty vector of samples.

map_to_new_backend(input, backend)

Move an array, or each array of a (q, p) named tuple, to a KernelAbstractions backend. An array that is already on backend is returned as it is.

map_to_type(input, T)

Convert an array, or each array of a (q, p) named tuple, to the element type T.

number_of_batches(dl, batch)

Compute the number of batches.

One epoch draws every start position of every parameter index once, in minibatches of the batch size, so the count is cld(n · n_params, batch_size) with n the number of start positions of one parameter index. No two indices are drawn twice, and the last minibatch holds what is left.

Examples

using ReducedComplexityModeling
using ReducedComplexityModeling: number_of_batches
import Random

Random.seed!(123)

dat = [1, 2, 3, 4, 5]
dl₁ = DataLoader(dat; autoencoder = false, suppress_info = true) # time series-like
dl₂ = DataLoader(dat; autoencoder = true, suppress_info = true) # autoencoder-like
batch = Batch(3)

batches₁ = batch(dl₁)
batches₂ = batch(dl₂)

@assert number_of_batches(dl₁, batch) == 2
@assert number_of_batches(dl₂, batch) == 2
@assert length.(batches₁) == [3, 1]
@assert length.(batches₂) == [3, 2]
@assert sort(collect(union(batches₁...))) == [(i, 1) for i in 1:4]
@assert sort(collect(union(batches₂...))) == [(1, i) for i in 1:5]

true

# output

true

Here we see that in the autoencoder case that last minibatch has an additional element.

project(rsol::GeometricSolution, model::ReducedBasisModel)

Map the solution rsol of a reduced problem (see compose) back to the full space. The result is the matrix model.reduced_basis * Q, where the columns of Q are the entries of rsol.q; its columns are the states in the full space.

read_parameters(h5, path = "/")
read_parameters(fpath)

Read the datasets of the group path of h5 into a named tuple. A child group is read as a Dict{String, Any}. With a file path, read the group "parameters" of fpath.

reduce!(model::ReducedBasisModel, k::Int, ::POD)
reduce!(model::ReducedBasisModel, k::Int, ::CotangentLiftSVD)
reduce!(model::ReducedBasisModel, k::Int, ::CotangentLiftEVD)
reduce!(model::ReducedBasisModel, k::Int, ::ComplexSVD)

Compute a reduced basis from the snapshot matrix of model, whose columns are the snapshots, store it in model.reduced_basis and return it. With POD, the basis is the matrix of the first k left singular vectors of the snapshot matrix. With CotangentLiftSVD(), CotangentLiftEVD() and ComplexSVD(), which the package does not export, the snapshot has 2n rows, q above p, and the basis is an orthonormal and symplectic 2n × 2k matrix, for k ≤ n. CotangentLiftSVD() and CotangentLiftEVD() give the cotangent lift of the first k left singular vectors of [q p]: the first takes them from the singular value decomposition of [q p], the second from the eigendecomposition of the Gram matrix q qᵀ + p pᵀ. The Gram matrix squares the condition number: in the floating-point type T of the basis, CotangentLiftEVD() resolves only the singular vectors whose singular values exceed about sqrt(eps(T)) times the largest, and its eigendecomposition of an n × n matrix takes O(n³) time and O(n²) memory. ComplexSVD() gives the real form of the first k left singular vectors of q + im * p. The three algorithms take a real snapshot: a complex one throws a MethodError. Where k exceeds the number of singular vectors, the basis is completed with orthonormal vectors. The basis has the floating-point type of the snapshot's element type: Float64 for an integer snapshot.

sample(sampler, parameters::NamedTuple)
sample(sampler, parameters::Parameter...)

Draws samples of parameters with sampler and returns them as a Table with one column per parameter. The column names are the keys of parameters, which must equal the parameter names. They are part of the type of a NamedTuple, so the first form infers a concrete Table. The second form builds the NamedTuple from the parameter names at run time and does not infer.

A sampler implements _columns(sampler, parameters::Tuple), which returns one sample vector per parameter.

save_parameters(h5, params; path = "/")
save_parameters(fpath, params)

Write each entry of the named tuple params as a dataset of the group path of h5. With a file path, open fpath and write to its group "parameters".

train!(model, …)

Train model in place and return the loss history.

The in-place companion of train: where train builds a model from the data and returns the fitted parameters, train! updates a model the caller already holds and returns the loss at each step of the fit, so that a caller can watch the fit. ReducedComplexityModeling defines no method of it; a package that owns a model provides one.

train(problem::TrainingProblem, method::Optim.FirstOrderOptimizer)

Fit the parameters of problem.data by minimising sum(abs2, problem.residual(data, θ)) over problem.parameters, with the first-order Optim method method, e.g. LBFGS().

The samples of the parameter space are starting points, not bounds: the fit starts from the sample whose loss is finite and smallest and may leave the box the samples span, so train returns the fitted parameters of the model and not a search over the box.

The result is a NamedTuple with the keys of problem.parameters.parameters and the element type of the samples. The fit has converged when the norm of the gradient of the loss falls to eps(T)^(2/3) times its norm at the start, or when a step changes the parameters by less than eps(T) relative to their size, with T that element type. Convergence is Optim's flag: an ErrorException is thrown when Optim reports that it did not converge, and a start at a stationary point ends the fit there. An exception raised by residual propagates to the caller.