quackamollie.model_manager.ollama.ollama_model_manager module

class quackamollie.model_manager.ollama.ollama_model_manager.OllamaQuackamollieModelManager[source]

Bases: MetaQuackamollieModelManager, MetaQuackamollieModel

Model manager managed by the QuackamollieModelManagerRegistry and serving models from the Ollama API

_abc_impl = <_abc._abc_data object>
_raw_model_list: None | List = None
classmethod astream_answer(content: str, chat_history: List, model_config: str | None = None, model_name: str | None = None, images_base64: List[str] | None = None, **kwargs) → AsyncIterable[Tuple[str, bool]][source]

Asynchronous iterator to stream the answer from a LLM model

Parameters:

  • content (str) – The new message content

  • chat_history (List) – A list of past messages formatted accordingly by model manager

  • model_config (Optional[str]) – Additional configuration given as a string through CLI or Telegram App Settings and retrieved from the database

  • model_name (Optional[str]) – Name of the model as listed by cls.get_model_list

  • images_base64 (Optional[List[str]]) – A list of images formatted as base64 strings

  • kwargs (kwargs) – Additional streaming arguments

Returns:

An asynchronous iterator giving a tuple containing the new chunk and a boolean indicating if the model is done or not

Return type:

AsyncIterable[Tuple[str, bool]]

families: List[ModelFamilyIcon] = [ModelFamilyIcon.LLAMA]
async classmethod get_model_class(model_name: str) → Type[MetaQuackamollieModel] | None[source]

Get the model class from the model name

Parameters:

model_name (str) – Name of the model as listed by cls.get_model_list

Returns:

A subclass of MetaQuackamollieModel

Return type:

Optional[Type[MetaQuackamollieModel]]

async classmethod get_model_families() → Dict[str, List[ModelFamilyIcon]][source]

Discover the models families available for the model manager at runtime asynchronously

Returns:

A dict with values the list of families indexed by model name

Return type:

Dict[str, List[ModelFamilyIcon]]

async classmethod get_model_list() → List[str][source]

Discover the models available for the model manager at runtime asynchronously

Returns:

A list of available models for the model manager

Return type:

List[str]

async classmethod get_raw_model_list() → List[source]
classmethod parse_chat_history(chat_messages: List[ChatMessage] | None) → List[Dict][source]

Parse the chat history given as a list of ChatMessage from the database model to a list compatible with the model manager’s models.

Parameters:

  • chat_messages – A list of ChatMessage from the database model

  • chat_messages – Optional[List[ChatMessage]]

Returns:

A list of messages formatted to be compatible with the model manager’s models.

Return type:

List[Dict]

classmethod reset()[source]

Reset the model manager dynamic fields to force reloading models. Be careful if used asynchronously