From f4488f4e1af21e360003b778b02dc013cf091e1d Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Wed, 13 Mar 2024 08:14:52 +0000 Subject: [PATCH] build based on f7cf63b --- dev/.documenter-siteinfo.json | 2 +- dev/alternatives/index.html | 2 +- dev/api/index.html | 50 +- dev/assets/documenter.js | 923 +++++++++++++++----------- dev/assets/themes/documenter-dark.css | 2 +- dev/debugging/index.html | 2 +- dev/examples/autodiff/index.html | 2 +- dev/examples/basics/index.html | 2 +- dev/examples/controlled/index.html | 20 +- dev/examples/interfaces/index.html | 2 +- dev/examples/temporal/index.html | 2 +- dev/examples/types/index.html | 2 +- dev/formulas/index.html | 2 +- dev/index.html | 2 +- dev/objects.inv | Bin 0 -> 1328 bytes 15 files changed, 588 insertions(+), 427 deletions(-) create mode 100644 dev/objects.inv diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 1e6a11bd..629bc8a4 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.1","generation_timestamp":"2024-02-29T14:41:00","documenter_version":"1.2.1"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.2","generation_timestamp":"2024-03-13T08:14:46","documenter_version":"1.3.0"}} \ No newline at end of file diff --git a/dev/alternatives/index.html b/dev/alternatives/index.html index aa9cbfb8..2f6a125d 100644 --- a/dev/alternatives/index.html +++ b/dev/alternatives/index.html @@ -1,2 +1,2 @@ -Alternatives · HiddenMarkovModels.jl
GitHub

Competitors

Julia

We compare features among the following Julia packages:

We discard MarkovModels.jl because its focus is GPU computation. There are also more generic packages for probabilistic programming, which are able to perform MCMC or variational inference (eg. Turing.jl) but we leave those aside.

HiddenMarkovModels.jlHMMBase.jlHMMGradients.jl
Algorithms[1]V, FB, BWV, FB, BWFB
Number typesanythingFloat64AbstractFloat
Observation typesanythingnumber or vectoranything
Observation distributionsDensityInterface.jlDistributions.jlmanual
Multiple sequencesyesnoyes
Priors / structurespossiblenopossible
Control dependencyyesnono
Automatic differentiationyesnoyes
Linear algebra speedupyesyesno
Numerical stabilityscaling+scaling+log
Very small probabilities

In all HMM algorithms, we work with probabilities that may become very small as time progresses. There are two main solutions for this problem: scaling and logarithmic computations. This package implements the Viterbi algorithm in log scale, but the other algorithms use scaling to exploit BLAS operations. As was done in HMMBase.jl, we enhance scaling with a division by the highest observation loglikelihood: instead of working with $b_{i,t} = \mathbb{P}(Y_t | X_t = i)$, we use $b_{i,t} / \max_i b_{i,t}$. See Formulas for details.

Python

We compare features among the following Python packages:

hmmlearnpomegranatedynamax
Algorithms[1]V, FB, BW, VIFB, BWFB, V, BW, GD
Number typesNumPy formatsPyTorch formatsJAX formats
Observation typesnumber or vectornumber or vectornumber or vector
Observation distributionshmmlearn cataloguepomegranate cataloguedynamax catalogue
Multiple sequencesyesyesyes
Priors / structuresyesnoyes
Control dependencynonoyes
Automatic differentiationnoyesyes
Linear algebra speedupyesyesyes
Numerical stabilityscaling / logloglog
  • 1V = Viterbi, FB = Forward-Backward, BW = Baum-Welch, VI = Variational Inference, GD = Gradient Descent
+Alternatives · HiddenMarkovModels.jl
GitHub

Competitors

Julia

We compare features among the following Julia packages:

We discard MarkovModels.jl because its focus is GPU computation. There are also more generic packages for probabilistic programming, which are able to perform MCMC or variational inference (eg. Turing.jl) but we leave those aside.

HiddenMarkovModels.jlHMMBase.jlHMMGradients.jl
Algorithms[1]V, FB, BWV, FB, BWFB
Number typesanythingFloat64AbstractFloat
Observation typesanythingnumber or vectoranything
Observation distributionsDensityInterface.jlDistributions.jlmanual
Multiple sequencesyesnoyes
Priors / structurespossiblenopossible
Control dependencyyesnono
Automatic differentiationyesnoyes
Linear algebra speedupyesyesno
Numerical stabilityscaling+scaling+log
Very small probabilities

In all HMM algorithms, we work with probabilities that may become very small as time progresses. There are two main solutions for this problem: scaling and logarithmic computations. This package implements the Viterbi algorithm in log scale, but the other algorithms use scaling to exploit BLAS operations. As was done in HMMBase.jl, we enhance scaling with a division by the highest observation loglikelihood: instead of working with $b_{i,t} = \mathbb{P}(Y_t | X_t = i)$, we use $b_{i,t} / \max_i b_{i,t}$. See Formulas for details.

Python

We compare features among the following Python packages:

hmmlearnpomegranatedynamax
Algorithms[1]V, FB, BW, VIFB, BWFB, V, BW, GD
Number typesNumPy formatsPyTorch formatsJAX formats
Observation typesnumber or vectornumber or vectornumber or vector
Observation distributionshmmlearn cataloguepomegranate cataloguedynamax catalogue
Multiple sequencesyesyesyes
Priors / structuresyesnoyes
Control dependencynonoyes
Automatic differentiationnoyesyes
Linear algebra speedupyesyesyes
Numerical stabilityscaling / logloglog
  • 1V = Viterbi, FB = Forward-Backward, BW = Baum-Welch, VI = Variational Inference, GD = Gradient Descent
diff --git a/dev/api/index.html b/dev/api/index.html index a45e4d82..0078244a 100644 --- a/dev/api/index.html +++ b/dev/api/index.html @@ -1,13 +1,13 @@ -API reference · HiddenMarkovModels.jl
GitHub

API reference

Sequence formatting

Most algorithms below ingest the data with two positional arguments obs_seq (mandatory) and control_seq (optional), and a keyword argument seq_ends (optional).

  • If the data consists of a single sequence, obs_seq and control_seq are the corresponding vectors of observations and controls, and you don't need to provide seq_ends.
  • If the data consists of multiple sequences, obs_seq and control_seq are concatenations of several vectors, whose end indices are given by seq_ends. Starting from separate sequences obs_seqs and control_seqs, you can run the following snippet:
obs_seq = reduce(vcat, obs_seqs)
+API reference · HiddenMarkovModels.jl
GitHub
API reference
Sequence formatting
Most algorithms below ingest the data with two positional arguments obs_seq (mandatory) and control_seq (optional), and a keyword argument seq_ends (optional).
  • If the data consists of a single sequence, obs_seq and control_seq are the corresponding vectors of observations and controls, and you don't need to provide seq_ends.
  • If the data consists of multiple sequences, obs_seq and control_seq are concatenations of several vectors, whose end indices are given by seq_ends. Starting from separate sequences obs_seqs and control_seqs, you can run the following snippet:
obs_seq = reduce(vcat, obs_seqs)
 control_seq = reduce(vcat, control_seqs)
-seq_ends = cumsum(length.(obs_seqs))
Types
HiddenMarkovModels.HMM — Type
struct HMM{V<:(AbstractVector), M<:(AbstractMatrix), VD<:(AbstractVector)} <: AbstractHMM
Basic implementation of an HMM.
Fields
  • init::AbstractVector: initial state probabilities
  • trans::AbstractMatrix: state transition probabilities
  • dists::AbstractVector: observation distributions
  • loginit::AbstractVector: logarithms of initial state probabilities
  • logtrans::AbstractMatrix: logarithms of state transition probabilities
source
Interface
HiddenMarkovModels.obs_distributions — Function
obs_distributions(hmm)
-obs_distributions(hmm, control)
Return a vector of observation distributions, one for each state of hmm (possibly when control is applied).
These distribution objects should implement
  • Random.rand(rng, dist) for sampling
  • DensityInterface.logdensityof(dist, obs) for inference
  • StatsAPI.fit!(dist, obs_seq, weight_seq) for learning
source
Utils
Base.rand — Function
rand([rng,] hmm, T)
-rand([rng,] hmm, control_seq)
Simulate hmm for T time steps, or when the sequence control_seq is applied.
Return a named tuple (; state_seq, obs_seq).
source
Base.eltype — Function
eltype(hmm, obs, control)
Return a type that can accommodate forward-backward computations for hmm on observations similar to obs.
It is typically a promotion between the element type of the initialization, the element type of the transition matrix, and the type of an observation logdensity evaluated at obs.
source
HiddenMarkovModels.seq_limits — Function
seq_limits(seq_ends, k)
-
Return a tuple (t1, t2) giving the begin and end indices of subsequence k within a set of sequences ending at seq_ends.
source
Inference
DensityInterface.logdensityof — Function
logdensityof(hmm)
Return the prior loglikelihood associated with the parameters of hmm.
source
logdensityof(hmm, obs_seq; ...)
+seq_ends = cumsum(length.(obs_seqs))
Types
HiddenMarkovModels.HMM — Type
struct HMM{V<:(AbstractVector), M<:(AbstractMatrix), VD<:(AbstractVector)} <: AbstractHMM
Basic implementation of an HMM.
Fields
  • init::AbstractVector: initial state probabilities
  • trans::AbstractMatrix: state transition probabilities
  • dists::AbstractVector: observation distributions
  • loginit::AbstractVector: logarithms of initial state probabilities
  • logtrans::AbstractMatrix: logarithms of state transition probabilities
source
Interface
HiddenMarkovModels.obs_distributions — Function
obs_distributions(hmm)
+obs_distributions(hmm, control)
Return a vector of observation distributions, one for each state of hmm (possibly when control is applied).
These distribution objects should implement
  • Random.rand(rng, dist) for sampling
  • DensityInterface.logdensityof(dist, obs) for inference
  • StatsAPI.fit!(dist, obs_seq, weight_seq) for learning
source
Utils
Base.rand — Function
rand([rng,] hmm, T)
+rand([rng,] hmm, control_seq)
Simulate hmm for T time steps, or when the sequence control_seq is applied.
Return a named tuple (; state_seq, obs_seq).
source
Base.eltype — Function
eltype(hmm, obs, control)
Return a type that can accommodate forward-backward computations for hmm on observations similar to obs.
It is typically a promotion between the element type of the initialization, the element type of the transition matrix, and the type of an observation logdensity evaluated at obs.
source
HiddenMarkovModels.seq_limits — Function
seq_limits(seq_ends, k)
+
Return a tuple (t1, t2) giving the begin and end indices of subsequence k within a set of sequences ending at seq_ends.
source
Inference
DensityInterface.logdensityof — Function
logdensityof(hmm)
Return the prior loglikelihood associated with the parameters of hmm.
source
logdensityof(hmm, obs_seq; ...)
 logdensityof(hmm, obs_seq, control_seq; seq_ends)
-
Run the forward algorithm to compute the loglikelihood of obs_seq for hmm, integrating over all possible state sequences.
source
HiddenMarkovModels.joint_logdensityof — Function
joint_logdensityof(hmm, obs_seq, state_seq; ...)
+
Run the forward algorithm to compute the loglikelihood of obs_seq for hmm, integrating over all possible state sequences.
source
HiddenMarkovModels.joint_logdensityof — Function
joint_logdensityof(hmm, obs_seq, state_seq; ...)
 joint_logdensityof(
     hmm,
     obs_seq,
@@ -15,13 +15,13 @@
     control_seq;
     seq_ends
 )
-
Run the forward algorithm to compute the the joint loglikelihood of obs_seq and state_seq for hmm.
source
HiddenMarkovModels.forward — Function
forward(hmm, obs_seq; ...)
+
Run the forward algorithm to compute the the joint loglikelihood of obs_seq and state_seq for hmm.
source
HiddenMarkovModels.forward — Function
forward(hmm, obs_seq; ...)
 forward(hmm, obs_seq, control_seq; seq_ends)
-
Apply the forward algorithm to infer the current state after sequence obs_seq for hmm.
Return a tuple (storage.α, storage.logL) where storage is of type ForwardStorage.
source
HiddenMarkovModels.viterbi — Function
viterbi(hmm, obs_seq; ...)
+
Apply the forward algorithm to infer the current state after sequence obs_seq for hmm.
Return a tuple (storage.α, storage.logL) where storage is of type ForwardStorage.
source
HiddenMarkovModels.viterbi — Function
viterbi(hmm, obs_seq; ...)
 viterbi(hmm, obs_seq, control_seq; seq_ends)
-
Apply the Viterbi algorithm to infer the most likely state sequence corresponding to obs_seq for hmm.
Return a tuple (storage.q, storage.logL) where storage is of type ViterbiStorage.
source
HiddenMarkovModels.forward_backward — Function
forward_backward(hmm, obs_seq; ...)
 forward_backward(hmm, obs_seq, control_seq; seq_ends)
-
Apply the forward-backward algorithm to infer the posterior state and transition marginals during sequence obs_seq for hmm.
Return a tuple (storage.γ, storage.logL) where storage is of type ForwardBackwardStorage.
source
Learning
HiddenMarkovModels.baum_welch — Function
baum_welch(hmm_guess, obs_seq; ...)
+
Apply the forward-backward algorithm to infer the posterior state and transition marginals during sequence obs_seq for hmm.
Return a tuple (storage.γ, storage.logL) where storage is of type ForwardBackwardStorage.
source
Learning
HiddenMarkovModels.baum_welch — Function
baum_welch(hmm_guess, obs_seq; ...)
 baum_welch(
     hmm_guess,
     obs_seq,
@@ -31,23 +31,23 @@
     max_iterations,
     loglikelihood_increasing
 )
-
Apply the Baum-Welch algorithm to estimate the parameters of an HMM on obs_seq, starting from hmm_guess.
Return a tuple (hmm_est, loglikelihood_evolution) where hmm_est is the estimated HMM and loglikelihood_evolution is a vector of loglikelihood values, one per iteration of the algorithm.
Keyword arguments
  • atol: minimum loglikelihood increase at an iteration of the algorithm (otherwise the algorithm is deemed to have converged)
  • max_iterations: maximum number of iterations of the algorithm
  • loglikelihood_increasing: whether to throw an error if the loglikelihood decreases
source
StatsAPI.fit! — Function
StatsAPI.fit!(
+
Apply the Baum-Welch algorithm to estimate the parameters of an HMM on obs_seq, starting from hmm_guess.
Return a tuple (hmm_est, loglikelihood_evolution) where hmm_est is the estimated HMM and loglikelihood_evolution is a vector of loglikelihood values, one per iteration of the algorithm.
Keyword arguments
  • atol: minimum loglikelihood increase at an iteration of the algorithm (otherwise the algorithm is deemed to have converged)
  • max_iterations: maximum number of iterations of the algorithm
  • loglikelihood_increasing: whether to throw an error if the loglikelihood decreases
source
StatsAPI.fit! — Function
StatsAPI.fit!(
     hmm, fb_storage::ForwardBackwardStorage,
     obs_seq, [control_seq]; seq_ends,
-)
Update hmm in-place based on information generated during forward-backward.
This function is allowed to reuse fb_storage as a scratch space, so its contents should not be trusted afterwards.
source
In-place versions
Forward
HiddenMarkovModels.ForwardStorage — Type
struct ForwardStorage{R}
Fields
Only the fields with a description are part of the public API.
  • α::Matrix: posterior last state marginals α[i] = ℙ(X[T]=i | Y[1:T])
  • logL::Vector: one loglikelihood per observation sequence
  • B::Matrix
  • c::Vector
source
Viterbi
HiddenMarkovModels.ViterbiStorage — Type
struct ViterbiStorage{R}
Fields
Only the fields with a description are part of the public API.
  • q::Vector{Int64}: most likely state sequence q[t] = argmaxᵢ ℙ(X[t]=i | Y[1:T])
  • logL::Vector: one joint loglikelihood per pair of observation sequence and most likely state sequence
  • logB::Matrix
  • ϕ::Matrix
  • ψ::Matrix{Int64}
source
Forward-backward
HiddenMarkovModels.ForwardBackwardStorage — Type
struct ForwardBackwardStorage{R, M<:AbstractArray{R, 2}}
Fields
Only the fields with a description are part of the public API.
  • γ::Matrix: posterior state marginals γ[i,t] = ℙ(X[t]=i | Y[1:T])
  • ξ::Vector{M} where {R, M<:AbstractMatrix{R}}: posterior transition marginals ξ[t][i,j] = ℙ(X[t]=i, X[t+1]=j | Y[1:T])
  • logL::Vector: one loglikelihood per observation sequence
  • B::Matrix
  • α::Matrix
  • c::Vector
  • β::Matrix
  • Bβ::Matrix
source
HiddenMarkovModels.initialize_forward_backward — Function
initialize_forward_backward(
+)
Update hmm in-place based on information generated during forward-backward.
This function is allowed to reuse fb_storage as a scratch space, so its contents should not be trusted afterwards.
source
In-place versions
Forward
HiddenMarkovModels.ForwardStorage — Type
struct ForwardStorage{R}
Fields
Only the fields with a description are part of the public API.
  • α::Matrix: posterior last state marginals α[i] = ℙ(X[T]=i | Y[1:T])
  • logL::Vector: one loglikelihood per observation sequence
  • B::Matrix
  • c::Vector
source
Viterbi
HiddenMarkovModels.ViterbiStorage — Type
struct ViterbiStorage{R}
Fields
Only the fields with a description are part of the public API.
  • q::Vector{Int64}: most likely state sequence q[t] = argmaxᵢ ℙ(X[t]=i | Y[1:T])
  • logL::Vector: one joint loglikelihood per pair of observation sequence and most likely state sequence
  • logB::Matrix
  • ϕ::Matrix
  • ψ::Matrix{Int64}
source
Forward-backward
HiddenMarkovModels.ForwardBackwardStorage — Type
struct ForwardBackwardStorage{R, M<:AbstractArray{R, 2}}
Fields
Only the fields with a description are part of the public API.
  • γ::Matrix: posterior state marginals γ[i,t] = ℙ(X[t]=i | Y[1:T])
  • ξ::Vector{M} where {R, M<:AbstractMatrix{R}}: posterior transition marginals ξ[t][i,j] = ℙ(X[t]=i, X[t+1]=j | Y[1:T])
  • logL::Vector: one loglikelihood per observation sequence
  • B::Matrix
  • α::Matrix
  • c::Vector
  • β::Matrix
  • Bβ::Matrix
source
Baum-Welch
Baum-Welch
Miscellaneous
HiddenMarkovModels.fit_in_sequence! — Function
fit_in_sequence!(dists, i, x, w)
-
Modify the i-th element of dists by fitting it to an observation sequence x with associated weight sequence w.
Default behavior:
fit!(dists[i], x, w)
Override for Distributions.jl (in the package extension)
dists[i] = fit(eltype(dists), turn_into_vector(x), w)
source
Internals
HiddenMarkovModels.LightDiagNormal — Type
struct LightDiagNormal{T1, T2, T3, V1<:AbstractArray{T1, 1}, V2<:AbstractArray{T2, 1}, V3<:AbstractArray{T3, 1}}
An HMMs-compatible implementation of a multivariate normal distribution with diagonal covariance, enabling allocation-free in-place estimation.
This is not part of the public API and is expected to change.
Fields
  • μ::AbstractVector: means
  • σ::AbstractVector: standard deviations
  • logσ::AbstractVector: log standard deviations
source
HiddenMarkovModels.LightCategorical — Type
struct LightCategorical{T1, T2, V1<:AbstractArray{T1, 1}, V2<:AbstractArray{T2, 1}}
An HMMs-compatible implementation of a discrete categorical distribution, enabling allocation-free in-place estimation.
This is not part of the public API and is expected to change.
Fields
  • p::AbstractVector: class probabilities
  • logp::AbstractVector: log class probabilities
source
HiddenMarkovModels.log_transition_matrix — Function
log_transition_matrix(hmm)
-log_transition_matrix(hmm, control)
Return the matrix of state transition log-probabilities for hmm (possibly when control is applied).
Falls back on transition_matrix.
source
HiddenMarkovModels.argmaxplus_transmul! — Function
argmaxplus_transmul!(y, ind, A, x)
Perform the in-place multiplication transpose(A) * x in the sense of max-plus algebra, store the result in y, and store the index of the maximum for each component of y in ind.
source
Index

+
source

Miscellaneous

HiddenMarkovModels.fit_in_sequence! — Function
fit_in_sequence!(dists, i, x, w)
+

Modify the i-th element of dists by fitting it to an observation sequence x with associated weight sequence w.

Default behavior:

fit!(dists[i], x, w)

Override for Distributions.jl (in the package extension)

dists[i] = fit(eltype(dists), turn_into_vector(x), w)
source

Internals

HiddenMarkovModels.LightDiagNormal — Type
struct LightDiagNormal{T1, T2, T3, V1<:AbstractArray{T1, 1}, V2<:AbstractArray{T2, 1}, V3<:AbstractArray{T3, 1}}

An HMMs-compatible implementation of a multivariate normal distribution with diagonal covariance, enabling allocation-free in-place estimation.

This is not part of the public API and is expected to change.

Fields

  • μ::AbstractVector: means

  • σ::AbstractVector: standard deviations

  • logσ::AbstractVector: log standard deviations

source
HiddenMarkovModels.LightCategorical — Type
struct LightCategorical{T1, T2, V1<:AbstractArray{T1, 1}, V2<:AbstractArray{T2, 1}}

An HMMs-compatible implementation of a discrete categorical distribution, enabling allocation-free in-place estimation.

This is not part of the public API and is expected to change.

Fields

  • p::AbstractVector: class probabilities

  • logp::AbstractVector: log class probabilities

source
HiddenMarkovModels.log_transition_matrix — Function
log_transition_matrix(hmm)
+log_transition_matrix(hmm, control)

Return the matrix of state transition log-probabilities for hmm (possibly when control is applied).

Falls back on transition_matrix.

source
HiddenMarkovModels.argmaxplus_transmul! — Function
argmaxplus_transmul!(y, ind, A, x)

Perform the in-place multiplication transpose(A) * x in the sense of max-plus algebra, store the result in y, and store the index of the maximum for each component of y in ind.

source

Index

diff --git a/dev/assets/documenter.js b/dev/assets/documenter.js index f5311607..c6562b55 100644 --- a/dev/assets/documenter.js +++ b/dev/assets/documenter.js @@ -4,7 +4,6 @@ requirejs.config({ 'highlight-julia': 'https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/languages/julia.min', 'headroom': 'https://cdnjs.cloudflare.com/ajax/libs/headroom/0.12.0/headroom.min', 'jqueryui': 'https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.13.2/jquery-ui.min', - 'minisearch': 'https://cdn.jsdelivr.net/npm/minisearch@6.1.0/dist/umd/index.min', 'katex-auto-render': 'https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.16.8/contrib/auto-render.min', 'jquery': 'https://cdnjs.cloudflare.com/ajax/libs/jquery/3.7.0/jquery.min', 'headroom-jquery': 'https://cdnjs.cloudflare.com/ajax/libs/headroom/0.12.0/jQuery.headroom.min', @@ -103,9 +102,10 @@ $(document).on("click", ".docstring header", function () { }); }); -$(document).on("click", ".docs-article-toggle-button", function () { +$(document).on("click", ".docs-article-toggle-button", function (event) { let articleToggleTitle = "Expand docstring"; let navArticleToggleTitle = "Expand all docstrings"; + let animationSpeed = event.noToggleAnimation ? 0 : 400; debounce(() => { if (isExpanded) { @@ -116,7 +116,7 @@ $(document).on("click", ".docs-article-toggle-button", function () { isExpanded = false; - $(".docstring section").slideUp(); + $(".docstring section").slideUp(animationSpeed); } else { $(this).removeClass("fa-chevron-down").addClass("fa-chevron-up"); $(".docstring-article-toggle-button") @@ -127,7 +127,7 @@ $(document).on("click", ".docs-article-toggle-button", function () { articleToggleTitle = "Collapse docstring"; navArticleToggleTitle = "Collapse all docstrings"; - $(".docstring section").slideDown(); + $(".docstring section").slideDown(animationSpeed); } $(this).prop("title", navArticleToggleTitle); @@ -224,224 +224,465 @@ $(document).ready(function () { }) //////////////////////////////////////////////////////////////////////////////// -require(['jquery', 'minisearch'], function($, minisearch) { - -// In general, most search related things will have "search" as a prefix. -// To get an in-depth about the thought process you can refer: https://hetarth02.hashnode.dev/series/gsoc +require(['jquery'], function($) { -let results = []; -let timer = undefined; +$(document).ready(function () { + let meta = $("div[data-docstringscollapsed]").data(); -let data = documenterSearchIndex["docs"].map((x, key) => { - x["id"] = key; // minisearch requires a unique for each object - return x; + if (meta?.docstringscollapsed) { + $("#documenter-article-toggle-button").trigger({ + type: "click", + noToggleAnimation: true, + }); + } }); -// list below is the lunr 2.1.3 list minus the intersect with names(Base) -// (all, any, get, in, is, only, which) and (do, else, for, let, where, while, with) -// ideally we'd just filter the original list but it's not available as a variable -const stopWords = new Set([ - "a", - "able", - "about", - "across", - "after", - "almost", - "also", - "am", - "among", - "an", - "and", - "are", - "as", - "at", - "be", - "because", - "been", - "but", - "by", - "can", - "cannot", - "could", - "dear", - "did", - "does", - "either", - "ever", - "every", - "from", - "got", - "had", - "has", - "have", - "he", - "her", - "hers", - "him", - "his", - "how", - "however", - "i", - "if", - "into", - "it", - "its", - "just", - "least", - "like", - "likely", - "may", - "me", - "might", - "most", - "must", - "my", - "neither", - "no", - "nor", - "not", - "of", - "off", - "often", - "on", - "or", - "other", - "our", - "own", - "rather", - "said", - "say", - "says", - "she", - "should", - "since", - "so", - "some", - "than", - "that", - "the", - "their", - "them", - "then", - "there", - "these", - "they", - "this", - "tis", - "to", - "too", - "twas", - "us", - "wants", - "was", - "we", - "were", - "what", - "when", - "who", - "whom", - "why", - "will", - "would", - "yet", - "you", - "your", -]); - -let index = new minisearch({ - fields: ["title", "text"], // fields to index for full-text search - storeFields: ["location", "title", "text", "category", "page"], // fields to return with search results - processTerm: (term) => { - let word = stopWords.has(term) ? null : term; - if (word) { - // custom trimmer that doesn't strip @ and !, which are used in julia macro and function names - word = word - .replace(/^[^a-zA-Z0-9@!]+/, "") - .replace(/[^a-zA-Z0-9@!]+$/, ""); - } +}) +//////////////////////////////////////////////////////////////////////////////// +require(['jquery'], function($) { - return word ?? null; - }, - // add . as a separator, because otherwise "title": "Documenter.Anchors.add!", would not find anything if searching for "add!", only for the entire qualification - tokenize: (string) => string.split(/[\s\-\.]+/), - // options which will be applied during the search - searchOptions: { - boost: { title: 100 }, - fuzzy: 2, +/* +To get an in-depth about the thought process you can refer: https://hetarth02.hashnode.dev/series/gsoc + +PSEUDOCODE: + +Searching happens automatically as the user types or adjusts the selected filters. +To preserve responsiveness, as much as possible of the slow parts of the search are done +in a web worker. Searching and result generation are done in the worker, and filtering and +DOM updates are done in the main thread. The filters are in the main thread as they should +be very quick to apply. This lets filters be changed without re-searching with minisearch +(which is possible even if filtering is on the worker thread) and also lets filters be +changed _while_ the worker is searching and without message passing (neither of which are +possible if filtering is on the worker thread) + +SEARCH WORKER: + +Import minisearch + +Build index + +On message from main thread + run search + find the first 200 unique results from each category, and compute their divs for display + note that this is necessary and sufficient information for the main thread to find the + first 200 unique results from any given filter set + post results to main thread + +MAIN: + +Launch worker + +Declare nonconstant globals (worker_is_running, last_search_text, unfiltered_results) + +On text update + if worker is not running, launch_search() + +launch_search + set worker_is_running to true, set last_search_text to the search text + post the search query to worker + +on message from worker + if last_search_text is not the same as the text in the search field, + the latest search result is not reflective of the latest search query, so update again + launch_search() + otherwise + set worker_is_running to false + + regardless, display the new search results to the user + save the unfiltered_results as a global + update_search() + +on filter click + adjust the filter selection + update_search() + +update_search + apply search filters by looping through the unfiltered_results and finding the first 200 + unique results that match the filters + + Update the DOM +*/ + +/////// SEARCH WORKER /////// + +function worker_function(documenterSearchIndex, documenterBaseURL, filters) { + importScripts( + "https://cdn.jsdelivr.net/npm/minisearch@6.1.0/dist/umd/index.min.js" + ); + + let data = documenterSearchIndex.map((x, key) => { + x["id"] = key; // minisearch requires a unique for each object + return x; + }); + + // list below is the lunr 2.1.3 list minus the intersect with names(Base) + // (all, any, get, in, is, only, which) and (do, else, for, let, where, while, with) + // ideally we'd just filter the original list but it's not available as a variable + const stopWords = new Set([ + "a", + "able", + "about", + "across", + "after", + "almost", + "also", + "am", + "among", + "an", + "and", + "are", + "as", + "at", + "be", + "because", + "been", + "but", + "by", + "can", + "cannot", + "could", + "dear", + "did", + "does", + "either", + "ever", + "every", + "from", + "got", + "had", + "has", + "have", + "he", + "her", + "hers", + "him", + "his", + "how", + "however", + "i", + "if", + "into", + "it", + "its", + "just", + "least", + "like", + "likely", + "may", + "me", + "might", + "most", + "must", + "my", + "neither", + "no", + "nor", + "not", + "of", + "off", + "often", + "on", + "or", + "other", + "our", + "own", + "rather", + "said", + "say", + "says", + "she", + "should", + "since", + "so", + "some", + "than", + "that", + "the", + "their", + "them", + "then", + "there", + "these", + "they", + "this", + "tis", + "to", + "too", + "twas", + "us", + "wants", + "was", + "we", + "were", + "what", + "when", + "who", + "whom", + "why", + "will", + "would", + "yet", + "you", + "your", + ]); + + let index = new MiniSearch({ + fields: ["title", "text"], // fields to index for full-text search + storeFields: ["location", "title", "text", "category", "page"], // fields to return with results processTerm: (term) => { let word = stopWords.has(term) ? null : term; if (word) { + // custom trimmer that doesn't strip @ and !, which are used in julia macro and function names word = word .replace(/^[^a-zA-Z0-9@!]+/, "") .replace(/[^a-zA-Z0-9@!]+$/, ""); + + word = word.toLowerCase(); } return word ?? null; }, + // add . as a separator, because otherwise "title": "Documenter.Anchors.add!", would not + // find anything if searching for "add!", only for the entire qualification tokenize: (string) => string.split(/[\s\-\.]+/), - }, -}); + // options which will be applied during the search + searchOptions: { + prefix: true, + boost: { title: 100 }, + fuzzy: 2, + }, + }); -index.addAll(data); + index.addAll(data); + + /** + * Used to map characters to HTML entities. + * Refer: https://github.com/lodash/lodash/blob/main/src/escape.ts + */ + const htmlEscapes = { + "&": "&", + "<": "<", + ">": ">", + '"': """, + "'": "'", + }; + + /** + * Used to match HTML entities and HTML characters. + * Refer: https://github.com/lodash/lodash/blob/main/src/escape.ts + */ + const reUnescapedHtml = /[&<>"']/g; + const reHasUnescapedHtml = RegExp(reUnescapedHtml.source); + + /** + * Escape function from lodash + * Refer: https://github.com/lodash/lodash/blob/main/src/escape.ts + */ + function escape(string) { + return string && reHasUnescapedHtml.test(string) + ? string.replace(reUnescapedHtml, (chr) => htmlEscapes[chr]) + : string || ""; + } -let filters = [...new Set(data.map((x) => x.category))]; -var modal_filters = make_modal_body_filters(filters); -var filter_results = []; + /** + * Make the result component given a minisearch result data object and the value + * of the search input as queryString. To view the result object structure, refer: + * https://lucaong.github.io/minisearch/modules/_minisearch_.html#searchresult + * + * @param {object} result + * @param {string} querystring + * @returns string + */ + function make_search_result(result, querystring) { + let search_divider = `
`; + let display_link = + result.location.slice(Math.max(0), Math.min(50, result.location.length)) + + (result.location.length > 30 ? "..." : ""); // To cut-off the link because it messes with the overflow of the whole div + + if (result.page !== "") { + display_link += ` (${result.page})`; + } -$(document).on("keyup", ".documenter-search-input", function (event) { - // Adding a debounce to prevent disruptions from super-speed typing! - debounce(() => update_search(filter_results), 300); + let textindex = new RegExp(`${querystring}`, "i").exec(result.text); + let text = + textindex !== null + ? result.text.slice( + Math.max(textindex.index - 100, 0), + Math.min( + textindex.index + querystring.length + 100, + result.text.length + ) + ) + : ""; // cut-off text before and after from the match + + text = text.length ? escape(text) : ""; + + let display_result = text.length + ? "..." + + text.replace( + new RegExp(`${escape(querystring)}`, "i"), // For first occurrence + '$&' + ) + + "..." + : ""; // highlights the match + + let in_code = false; + if (!["page", "section"].includes(result.category.toLowerCase())) { + in_code = true; + } + + // We encode the full url to escape some special characters which can lead to broken links + let result_div = ` + +
+
${escape(result.title)}
+
${result.category}
+
+

+ ${display_result} +

+
+ ${display_link} +
+ + ${search_divider} + `; + + return result_div; + } + + self.onmessage = function (e) { + let query = e.data; + let results = index.search(query, { + filter: (result) => { + // Only return relevant results + return result.score >= 1; + }, + }); + + // Pre-filter to deduplicate and limit to 200 per category to the extent + // possible without knowing what the filters are. + let filtered_results = []; + let counts = {}; + for (let filter of filters) { + counts[filter] = 0; + } + let present = {}; + + for (let result of results) { + cat = result.category; + cnt = counts[cat]; + if (cnt < 200) { + id = cat + "---" + result.location; + if (present[id]) { + continue; + } + present[id] = true; + filtered_results.push({ + location: result.location, + category: cat, + div: make_search_result(result, query), + }); + } + } + + postMessage(filtered_results); + }; +} + +// `worker = Threads.@spawn worker_function(documenterSearchIndex)`, but in JavaScript! +const filters = [ + ...new Set(documenterSearchIndex["docs"].map((x) => x.category)), +]; +const worker_str = + "(" + + worker_function.toString() + + ")(" + + JSON.stringify(documenterSearchIndex["docs"]) + + "," + + JSON.stringify(documenterBaseURL) + + "," + + JSON.stringify(filters) + + ")"; +const worker_blob = new Blob([worker_str], { type: "text/javascript" }); +const worker = new Worker(URL.createObjectURL(worker_blob)); + +/////// SEARCH MAIN /////// + +// Whether the worker is currently handling a search. This is a boolean +// as the worker only ever handles 1 or 0 searches at a time. +var worker_is_running = false; + +// The last search text that was sent to the worker. This is used to determine +// if the worker should be launched again when it reports back results. +var last_search_text = ""; + +// The results of the last search. This, in combination with the state of the filters +// in the DOM, is used compute the results to display on calls to update_search. +var unfiltered_results = []; + +// Which filter is currently selected +var selected_filter = ""; + +$(document).on("input", ".documenter-search-input", function (event) { + if (!worker_is_running) { + launch_search(); + } }); +function launch_search() { + worker_is_running = true; + last_search_text = $(".documenter-search-input").val(); + worker.postMessage(last_search_text); +} + +worker.onmessage = function (e) { + if (last_search_text !== $(".documenter-search-input").val()) { + launch_search(); + } else { + worker_is_running = false; + } + + unfiltered_results = e.data; + update_search(); +}; + $(document).on("click", ".search-filter", function () { if ($(this).hasClass("search-filter-selected")) { - $(this).removeClass("search-filter-selected"); + selected_filter = ""; } else { - $(this).addClass("search-filter-selected"); + selected_filter = $(this).text().toLowerCase(); } - // Adding a debounce to prevent disruptions from crazy clicking! - debounce(() => get_filters(), 300); + // This updates search results and toggles classes for UI: + update_search(); }); -/** - * A debounce function, takes a function and an optional timeout in milliseconds - * - * @function callback - * @param {number} timeout - */ -function debounce(callback, timeout = 300) { - clearTimeout(timer); - timer = setTimeout(callback, timeout); -} - /** * Make/Update the search component - * - * @param {string[]} selected_filters */ -function update_search(selected_filters = []) { - let initial_search_body = ` -
Type something to get started!
- `; - +function update_search() { let querystring = $(".documenter-search-input").val(); if (querystring.trim()) { - results = index.search(querystring, { - filter: (result) => { - // Filtering results - if (selected_filters.length === 0) { - return result.score >= 1; - } else { - return ( - result.score >= 1 && selected_filters.includes(result.category) - ); - } - }, - }); + if (selected_filter == "") { + results = unfiltered_results; + } else { + results = unfiltered_results.filter((result) => { + return selected_filter == result.category.toLowerCase(); + }); + } let search_result_container = ``; + let modal_filters = make_modal_body_filters(); let search_divider = `
`; if (results.length) { @@ -449,19 +690,23 @@ function update_search(selected_filters = []) { let count = 0; let search_results = ""; - results.forEach(function (result) { - if (result.location) { - // Checking for duplication of results for the same page - if (!links.includes(result.location)) { - search_results += make_search_result(result, querystring); - count++; - } - + for (var i = 0, n = results.length; i < n && count < 200; ++i) { + let result = results[i]; + if (result.location && !links.includes(result.location)) { + search_results += result.div; + count++; links.push(result.location); } - }); + } - let result_count = `
${count} result(s)
`; + if (count == 1) { + count_str = "1 result"; + } else if (count == 200) { + count_str = "200+ results"; + } else { + count_str = count + " results"; + } + let result_count = `
${count_str}
`; search_result_container = `
@@ -490,125 +735,37 @@ function update_search(selected_filters = []) { $(".search-modal-card-body").html(search_result_container); } else { - filter_results = []; - modal_filters = make_modal_body_filters(filters, filter_results); - if (!$(".search-modal-card-body").hasClass("is-justify-content-center")) { $(".search-modal-card-body").addClass("is-justify-content-center"); } - $(".search-modal-card-body").html(initial_search_body); + $(".search-modal-card-body").html(` +
Type something to get started!
+ `); } } /** * Make the modal filter html * - * @param {string[]} filters - * @param {string[]} selected_filters * @returns string */ -function make_modal_body_filters(filters, selected_filters = []) { - let str = ``; - - filters.forEach((val) => { - if (selected_filters.includes(val)) { - str += `${val}`; - } else { - str += `${val}`; - } - }); +function make_modal_body_filters() { + let str = filters + .map((val) => { + if (selected_filter == val.toLowerCase()) { + return `${val}`; + } else { + return `${val}`; + } + }) + .join(""); - let filter_html = ` + return `
Filters: ${str} -
- `; - - return filter_html; -} - -/** - * Make the result component given a minisearch result data object and the value of the search input as queryString. - * To view the result object structure, refer: https://lucaong.github.io/minisearch/modules/_minisearch_.html#searchresult - * - * @param {object} result - * @param {string} querystring - * @returns string - */ -function make_search_result(result, querystring) { - let search_divider = `
`; - let display_link = - result.location.slice(Math.max(0), Math.min(50, result.location.length)) + - (result.location.length > 30 ? "..." : ""); // To cut-off the link because it messes with the overflow of the whole div - - if (result.page !== "") { - display_link += ` (${result.page})`; - } - - let textindex = new RegExp(`\\b${querystring}\\b`, "i").exec(result.text); - let text = - textindex !== null - ? result.text.slice( - Math.max(textindex.index - 100, 0), - Math.min( - textindex.index + querystring.length + 100, - result.text.length - ) - ) - : ""; // cut-off text before and after from the match - - let display_result = text.length - ? "..." + - text.replace( - new RegExp(`\\b${querystring}\\b`, "i"), // For first occurrence - '$&' - ) + - "..." - : ""; // highlights the match - - let in_code = false; - if (!["page", "section"].includes(result.category.toLowerCase())) { - in_code = true; - } - - // We encode the full url to escape some special characters which can lead to broken links - let result_div = ` - -
-
${result.title}
-
${result.category}
-
-

- ${display_result} -

-
- ${display_link} -
- - ${search_divider} - `; - - return result_div; -} - -/** - * Get selected filters, remake the filter html and lastly update the search modal - */ -function get_filters() { - let ele = $(".search-filters .search-filter-selected").get(); - filter_results = ele.map((x) => $(x).text().toLowerCase()); - modal_filters = make_modal_body_filters(filters, filter_results); - update_search(filter_results); +
`; } }) @@ -635,103 +792,107 @@ $(document).ready(function () { //////////////////////////////////////////////////////////////////////////////// require(['jquery'], function($) { -let search_modal_header = ` -
-
-

- - - - -

-
-
- -
-
-`; - -let initial_search_body = ` -
Type something to get started!
-`; - -let search_modal_footer = ` - -`; - -$(document.body).append( - ` - +)
((nothing, nothing, nothing, nothing),)

Once again we can check the results.

∇parameters_enzyme ≈ ∇parameters_forwarddiff
true
∇obs_enzyme ≈ ∇obs_forwarddiff
true
∇control_enzyme ≈ ∇control_forwarddiff
true

For increased efficiency, we could provide temporary storage to Enzyme.jl in order to avoid allocations. This requires going one level deeper and leveraging the in-place HiddenMarkovModels.forward! function.

Gradient methods

Once we have gradients of the loglikelihood, it is a natural idea to perform gradient descent in order to fit the parameters of a custom HMM. However, there are two caveats we must keep in mind.

First, computing a gradient essentially requires running the forward-backward algorithm, which means it is expensive. Given the output of forward-backward, if there is a way to perform a more accurate parameter update (like going straight to the maximum likelihood value), it is probably worth it. That is what we show in the other tutorials with the reimplementation of the fit! method.

Second, HMM parameters live in a constrained space, which calls for a projected gradient descent. Most notably, the transition matrix must be stochastic, and the orthogonal projection onto this set (the Birkhoff polytope) is not easy to obtain.

Still, first order optimization can be relevant when we lack explicit formulas for maximum likelihood.

This page was generated using Literate.jl.

diff --git a/dev/examples/basics/index.html b/dev/examples/basics/index.html index 36ea5076..7194780b 100644 --- a/dev/examples/basics/index.html +++ b/dev/examples/basics/index.html @@ -65,4 +65,4 @@ [-0.50037, -0.782225] [-0.5, -0.8] [0.519063, 0.80974] [0.5, 0.8]
hcat(initialization(hmm_est_concat), initialization(hmm))
2×2 Matrix{Float64}:
  0.603183  0.6
- 0.396817  0.4

This page was generated using Literate.jl.

+ 0.396817 0.4

This page was generated using Literate.jl.

diff --git a/dev/examples/controlled/index.html b/dev/examples/controlled/index.html index 26ffac8d..761dd412 100644 --- a/dev/examples/controlled/index.html +++ b/dev/examples/controlled/index.html @@ -28,7 +28,7 @@ obs_seq = reduce(vcat, obs_seqs) control_seq = reduce(vcat, control_seqs) -seq_ends = cumsum(length.(obs_seqs));

Inference

Not much changes from the case with simple time dependency.

best_state_seq, _ = viterbi(hmm, obs_seq, control_seq; seq_ends)
([1, 1, 1, 1, 2, 2, 1, 1, 1, 1  …  1, 2, 2, 2, 1, 1, 1, 1, 2, 2], [-201.25159116814518, -360.82665432971424, -277.4327634821748, -372.49816864565094, -362.9895454729635, -260.59221112575653, -285.69652038135723, -296.8427840039318, -183.29484442508624, -235.41409942097843  …  -226.26185177959803, -203.18854385894028, -201.84612889178234, -310.1105771063101, -294.57801782364277, -367.66262590197096, -347.49662965885045, -222.31914482273908, -210.69547749764473, -211.90166492551674])

Learning

Once more, we override the fit! function. The state-related parameters are estimated in the standard way. Meanwhile, the observation coefficients are given by the formula for weighted least squares.

function StatsAPI.fit!(
+seq_ends = cumsum(length.(obs_seqs));

Inference

Not much changes from the case with simple time dependency.

best_state_seq, _ = viterbi(hmm, obs_seq, control_seq; seq_ends)
([1, 1, 1, 1, 1, 1, 1, 2, 2, 2  …  1, 2, 2, 2, 2, 2, 1, 1, 1, 1], [-349.502177419659, -257.70042407152, -302.88950114779163, -260.4225973535134, -304.97435396984326, -348.51412618987183, -358.00929111415724, -374.6973776249435, -325.2972267926067, -308.9787025279921  …  -375.5538113088102, -277.91456311196276, -175.4321193642634, -189.1724107050311, -289.4696815858785, -353.47003026267737, -196.68841746439034, -190.43637168540363, -223.26685236737583, -294.24525004256816])

Learning

Once more, we override the fit! function. The state-related parameters are estimated in the standard way. Meanwhile, the observation coefficients are given by the formula for weighted least squares.

function StatsAPI.fit!(
     hmm::ControlledGaussianHMM{T},
     fb_storage::HMMs.ForwardBackwardStorage,
     obs_seq::AbstractVector,
@@ -60,17 +60,17 @@
 trans_guess = [0.6 0.4; 0.3 0.7]
 dist_coeffs_guess = [-1.1 * ones(d), 1.1 * ones(d)]
 hmm_guess = ControlledGaussianHMM(init_guess, trans_guess, dist_coeffs_guess);
hmm_est, loglikelihood_evolution = baum_welch(hmm_guess, obs_seq, control_seq; seq_ends)
-first(loglikelihood_evolution), last(loglikelihood_evolution)
(-26194.850025775195, -25858.848500391945)

How did we perform?

cat(hmm_est.trans, hmm.trans; dims=3)
2×2×2 Array{Float64, 3}:
+first(loglikelihood_evolution), last(loglikelihood_evolution)
(-26563.880396597342, -26164.022408995876)

How did we perform?

cat(hmm_est.trans, hmm.trans; dims=3)
2×2×2 Array{Float64, 3}:
 [:, :, 1] =
- 0.695077  0.304923
- 0.210996  0.789004
+ 0.698916  0.301084
+ 0.200885  0.799115
 
 [:, :, 2] =
  0.7  0.3
  0.2  0.8
hcat(hmm_est.dist_coeffs[1], hmm.dist_coeffs[1])
3×2 Matrix{Float64}:
- -1.00541   -1.0
- -0.989645  -1.0
- -0.997075  -1.0
hcat(hmm_est.dist_coeffs[2], hmm.dist_coeffs[2])
3×2 Matrix{Float64}:
- 1.00297   1.0
- 0.993125  1.0
- 1.00063   1.0

This page was generated using Literate.jl.

+ -1.02165 -1.0 + -0.990595 -1.0 + -0.997056 -1.0
hcat(hmm_est.dist_coeffs[2], hmm.dist_coeffs[2])
3×2 Matrix{Float64}:
+ 0.967925  1.0
+ 0.993068  1.0
+ 0.992708  1.0

This page was generated using Literate.jl.

diff --git a/dev/examples/interfaces/index.html b/dev/examples/interfaces/index.html index 90f443f8..5c33b323 100644 --- a/dev/examples/interfaces/index.html +++ b/dev/examples/interfaces/index.html @@ -87,4 +87,4 @@ [:, :, 2] = 0.594827 0.405173 - 0.464871 0.535129
std(vec(transition_matrix(hmm_est))) < std(vec(transition_matrix(hmm)))
true

This page was generated using Literate.jl.

+ 0.464871 0.535129
std(vec(transition_matrix(hmm_est))) < std(vec(transition_matrix(hmm)))
true

This page was generated using Literate.jl.

diff --git a/dev/examples/temporal/index.html b/dev/examples/temporal/index.html index 717c3769..7c2ab937 100644 --- a/dev/examples/temporal/index.html +++ b/dev/examples/temporal/index.html @@ -97,4 +97,4 @@ -1.00183 -1.0 -2.01198 -2.0
map(mean, hcat(obs_distributions(hmm_est, 2), obs_distributions(hmm, 2)))
2×2 Matrix{Float64}:
  0.972284  1.0
- 2.01909   2.0

This page was generated using Literate.jl.

+ 2.01909 2.0

This page was generated using Literate.jl.

diff --git a/dev/examples/types/index.html b/dev/examples/types/index.html index 819e7f7b..535beb98 100644 --- a/dev/examples/types/index.html +++ b/dev/examples/types/index.html @@ -38,4 +38,4 @@ first(loglikelihood_evolution), last(loglikelihood_evolution)
(-1649.3075577938478, -1635.2699650704797)

The estimated model has kept the same sparsity pattern as the guess.

transition_matrix(hmm_est)
3×3 SparseArrays.SparseMatrixCSC{Float64, Int64} with 6 stored entries:
  0.718863  0.281137   ⋅ 
   ⋅        0.659587  0.340413
- 0.287745   ⋅        0.712255

Another useful array type is StaticArrays.jl, which reduces allocations for small state spaces.

This page was generated using Literate.jl.

+ 0.287745 ⋅ 0.712255

Another useful array type is StaticArrays.jl, which reduces allocations for small state spaces.

This page was generated using Literate.jl.

diff --git a/dev/formulas/index.html b/dev/formulas/index.html index 69935a46..3641887b 100644 --- a/dev/formulas/index.html +++ b/dev/formulas/index.html @@ -56,4 +56,4 @@ \frac{\partial \log \mathcal{L}}{\partial a_{i,j}} &= \sum_{t=1}^{T-1} \bar{\alpha}_{i,t} \frac{b_{j,t+1}}{m_{t+1}} \bar{\beta}_{j,t+1} \\ \frac{\partial \log \mathcal{L}}{\partial \log b_{j,1}} &= \pi_j \frac{b_{j,1}}{m_1} \bar{\beta}_{j,1} = \frac{\bar{\alpha}_{j,1} \bar{\beta}_{j,1}}{c_1} = \gamma_{j,1} \\ \frac{\partial \log \mathcal{L}}{\partial \log b_{j,t}} &= \sum_{i=1}^N \bar{\alpha}_{i,t-1} a_{i,j,t-1} \frac{b_{j,t}}{m_t} \bar{\beta}_{j,t} = \frac{\bar{\alpha}_{j,t} \bar{\beta}_{j,t}}{c_t} = \gamma_{j,t} -\end{align*}\]

Bibliography

+\end{align*}\]

Bibliography

diff --git a/dev/index.html b/dev/index.html index bfd0c883..defc36ab 100644 --- a/dev/index.html +++ b/dev/index.html @@ -3,4 +3,4 @@ init = [0.6, 0.4] trans = [0.7 0.3; 0.2 0.8] dists = [Normal(-1.0), Normal(1.0)] -hmm = HMM(init, trans, dists)

Take a look at the documentation to know what to do next!

Some background

Hidden Markov Models (HMMs) are a widely used modeling framework in signal processing, bioinformatics and plenty of other fields. They explain an observation sequence $(Y_t)$ by assuming the existence of a latent Markovian state sequence $(X_t)$ whose current value determines the distribution of observations. In some scenarios, the state and the observation sequence are also allowed to depend on a known control sequence $(U_t)$. Each of the problems below has an efficient solution algorithm, available here:

ProblemGoalAlgorithm
EvaluationLikelihood of the observation sequenceForward
FilteringLast state marginalsForward
SmoothingAll state marginalsForward-backward
DecodingMost likely state sequenceViterbi
LearningMaximum likelihood parameterBaum-Welch

Take a look at this tutorial to know more about the math:

A tutorial on hidden Markov models and selected applications in speech recognition, Rabiner (1989)

Main features

This package is generic. Observations can be arbitrary Julia objects, not just scalars or arrays. Number types are not restricted to floating point, which enables automatic differentiation. Time-dependent or controlled HMMs are supported out of the box.

This package is fast. All the inference functions have allocation-free versions, which leverage efficient linear algebra subroutines. We will include extensive benchmarks against Julia and Python competitors.

This package is reliable. It gives the same results as the previous reference package up to numerical accuracy. The test suite incorporates quality checks as well as type stability and allocation analysis.

Contributing

If you spot a bug or want to ask about a new feature, please open an issue on the GitHub repository. Once the issue receives positive feedback, feel free to try and fix it with a pull request that follows the BlueStyle guidelines.

Acknowledgements

A big thank you to Maxime Mouchet and Jacob Schreiber, the respective lead devs of alternative packages HMMBase.jl and pomegranate, for their help and advice. Logo by Clément Mantoux based on a portrait of Andrey Markov.

+hmm = HMM(init, trans, dists)

Take a look at the documentation to know what to do next!

Some background

Hidden Markov Models (HMMs) are a widely used modeling framework in signal processing, bioinformatics and plenty of other fields. They explain an observation sequence $(Y_t)$ by assuming the existence of a latent Markovian state sequence $(X_t)$ whose current value determines the distribution of observations. In some scenarios, the state and the observation sequence are also allowed to depend on a known control sequence $(U_t)$. Each of the problems below has an efficient solution algorithm, available here:

ProblemGoalAlgorithm
EvaluationLikelihood of the observation sequenceForward
FilteringLast state marginalsForward
SmoothingAll state marginalsForward-backward
DecodingMost likely state sequenceViterbi
LearningMaximum likelihood parameterBaum-Welch

Take a look at this tutorial to know more about the math:

A tutorial on hidden Markov models and selected applications in speech recognition, Rabiner (1989)

Main features

This package is generic. Observations can be arbitrary Julia objects, not just scalars or arrays. Number types are not restricted to floating point, which enables automatic differentiation. Time-dependent or controlled HMMs are supported out of the box.

This package is fast. All the inference functions have allocation-free versions, which leverage efficient linear algebra subroutines. We will include extensive benchmarks against Julia and Python competitors.

This package is reliable. It gives the same results as the previous reference package up to numerical accuracy. The test suite incorporates quality checks as well as type stability and allocation analysis.

Contributing

If you spot a bug or want to ask about a new feature, please open an issue on the GitHub repository. Once the issue receives positive feedback, feel free to try and fix it with a pull request that follows the BlueStyle guidelines.

Acknowledgements

A big thank you to Maxime Mouchet and Jacob Schreiber, the respective lead devs of alternative packages HMMBase.jl and pomegranate, for their help and advice. Logo by Clément Mantoux based on a portrait of Andrey Markov.

diff --git a/dev/objects.inv b/dev/objects.inv new file mode 100644 index 0000000000000000000000000000000000000000..013d08951b02c9e452d87c24795d78d70285701a GIT binary patch literal 1328 zcmV-01<(2;AX9K?X>NERX>N99Zgg*Qc_4OWa&u{KZXhxWBOp+6Z)#;@bUGkNX=G$& zZcSlwYj1WMd?av*PJ zAarPHb0B7EY-J#6b0A}HZE$jBb8}^6Aa!$TZf78RY-wUH3V58QS50%{HVnPVh;sOBWlZv{^kDl{z6$W8=eF} zy?I3+yXV1ikQdr;Ienrx;-Ml3nnD3suu2X+KGDh=EhVIr5qk5A-ZsyZVR&VADmI(R zAgyslul*C@g&4ZxM>KwYqy{!%FfD_XPD+>} zD`FcYMI`N!{dx0(V4DkWN?)b~RUS1&RU zs-r7L^JwbO)S+pI#!6TrF;C`PgjVHb2U{=NdJ#WIte>MmB7@;`?HrHaQ^L{6HVsW{ zE{^#+Bv=EWrvUO9ti`#&8|KEsn9ANxy`4BYxtxC|KN#UufWA~Y;01BXa5V6~rQRP$ z>E{;Foj2{gn>LELvhBbR6J}&xNh>f7R>HrPV|s`=UdR#DLTCpe-?$P|@&zsSh;fha zkW!Un0Ba`V67@QNH$mk0N(z2y_MbuEiK@zDFsxrhXQrh_&;~*4Z^=!+^UgI3I}E5t zDFb7)DbHM9<3}4mvgSgu4e$vaalUA!t(9CSuS)Ncu^Hr`?ARSe5!ry<=OeK@N&#hY zxS-BAo{6}Zy_mox?87hj z)ETJN^f(?_i!uRZoB~} z{b=upq^xrG3^LgUD*W3D8LIyN?C^GOHzJloXNDKsLwLSfv%wUo)k=gI$RFqx;N6Y8@NBXm|ka0OYgbsX#`uyq@(?GQ+Z4 z2=x%BJ8&gIWk{MA`Ds(B#N|f^ot>CVp#;u>6yG88in{=8xJskK=s4OzvJ_|WZy6y$ z7XYtgY&=h_+W?4i9ld~%I%`f6E__tK*1Rn^5qTRKy)UCg%kqaQ;3d$9pFf6E>)Oxj zkn{8ULu6ZfivpJEX0gv}hri>f9p$+do#H}8xo|j$mAsgUuvmQ5zA4Pk;~3VYQ8OQ8 zUH<$Idfj+JQRf-V?BA~0SaZ|$?t?TP^%o?$d?tNaH~ylQmB&q%^S44)WivELV9hTQs%kG`v*%$Y2 mwNP~zif!;_dQQG)(J$RJsI`$4xv;_On6WQH(Ek8%z!Tas`-hqU literal 0 HcmV?d00001