Genie Discord forum

coming from Flask, I am not sure here how to load something (an LLM) when the API is launched... It cannot be handled in the route otherwise every call will hit that, but I am not sure where I should load my model... any hint to documentation that I might have missed is more than welcome!

You can load it when you load the API module, before the route definitions. This is how I'm doing it in an app I'm writing now (see image)

see the includeand @loadstatements. You could have something like

module API
LLM = load_my_llm()
route("/llm") do
  process_query()
end
end

thank you! it is indeed simpler than I thought...

beware of globals though - make it a const or a Ref

mm not sure I understand, I am really a newbie in julia

here: https://docs.julialang.org/en/v1/manual/performance-tips/#Avoid-untyped-global-variables