Journal

TODO: llm-assistant

• Add tests for sync

• Go through the how-to guides of pytest ant check if I can improve some of my practices.

• Implement python package generator from collection of llm-assistant custom prompts.

2025-02-12

• In NewInstance: Output the variable names such that one can know which variables to enter.

• The CLI is kind of slow. This could be improved by changing how I do imports. [abandonned]

• prompt-manager push: It is not working if the repository has no commits or tags yet. [abandonned]

• Instead of using print to print information, use log with the info flag and direct the output to stderr.

• The integration test is failing in the CI because I am executing only the api test and the coverage limit of 75% is not reached.
• prompt-manager: Allow command run-test to execute over two versions given as parameters.
• prompt-manager: Update documentation regarding the push procedure and how prompt-versioning is supposed to work.
• I updated correct-text v2.0. It now uses xml tags to specify input parameters. This seems to reduce ambiguities and improvements were noticed in tests.
• About the use of json in test format. One disadvantage is that some of the outputs could be multiline strings or even use another format such as markdown. JSON strings are not very friendly.
• Regarding the tests input, I can update the code to accept json objects as input, which will reduce the fatigue to interpret some tests.

2025-02-11

• Add tests to each of the prompts in danoan-prompts.
• Normalized branches and versions in danoan-prompts.

2025-01-31

• IMPORTANT: Remember to use the integration-test key when running the tests and generating the cache. Otherwise, there will be a fail in the github pipeline.

• Write about how to correctly use the llm assistant environment variable in order to have multiple llm-assistant configurations (instances). For example, by modifying the activate.sh of a virtual environment one can change the llm assistant environment variable. The same is valid for word-guru. By the way, write about how the two systems interact with.

• Update project to use python3.12
• Improved llm-assistant setup init command
• Release version 0.4

2024-10-31

• In LoadInstance: Check if there is any instance, otherwise the program aborts with an error.
• Write design documentation for TaskRunner.
• Add tests for prompt api
• Implement functional version of correct-markdown pipeline.
• Refactor Makefile pipeline for correct-markdown. Try to find a pattern of operating that is general enough to replicate by other workflows.
• Write a README.md for the workflow.

2024-10-01

• Implemented run-tests and regenerate-tests
• Refactor prompt-cli commands

2024-09-28

• Refactor the configuration file.
  • Separated Runner and PromptRepositoryConfiguration
  • Both configurations lie in the same toml file
• Implemented sync function
  • sync local folder and configuration file

The prompt sync functionality

• LLMA runs prompts from a local folder.
• This local folder can be manually configured via a configuration parameter.
• This folder can also be sync with git repositories.
• When run is executed, no prior sync operation is executed, excepted if the prompt folder is not found. In this case we check if a sync will solve our issue.

LLM-assistant uses

• To start a chat session using versioned prompts.
• To build pipelines. Prompts and their execution are handled by llm-assistant. (enchain).
• To use as a library. We can pass the configuration file as a parameter and then obtain the same functionality via code (word-guru).

2024-09-24

• Create separated branches for each version of the prompts.

2024-09-22

• Released v0.2.0 of llm-assistant

• Started working on v0.3.0 of llm-assistant

  • Search configuration file in the working directory
  • Fallback to location specified in the environment variable
  • Update configuration file to specify prompt repository and versions

• Integrated prompt-evaluator within llm-assistant

2024-09-21

• Tried a file organization dividing input/output of prompt and script related assets:
  • prompt-assets
    • input
    • output
  • script-assets
    • input
    • output
• I found it confusing because we may have targets using inputs from prompts and scripts at the same time. I think it is better to isolate the targets in their own folders, even if that means having some files duplicated.

2024-08-01

In linux systems that use Wayland, pynput won't work properly. I was expecting that by running under the XWayland (an emulator of X11) I would be able to listen the keyboard events with pynput, but after several attempts I was not able to make it work.

I tried to force using x11 by setting up the environment variable GDK_BACKEND to x11 but it didn't work.

Finally, I decided to fallback to the mechanism in pynput that uses the uinput library. The only issue here is that I need sudo access.

PYNPUT_BACKEND_KEYBOARD=uinput sudo -E ./.venv/bin/llm-assistant session

References

• Wayland backend for pynput(open issue)
• Instruct pynput to use uinput

2024-07-31

The X Window System

It is a client-server system commonly used in Unix-like systems to communicate with hardware such as display, keyboard and mouse. It provides a low-level api for GUI libraries such as GTK+ and Qt.

The X server running on a machine owns the graphical resources of that machine ( keyboard, mouse, display). Clients are started by applications that want to use the resources of the server to render in the display or to monitor events in the mouse or keyboard.

Clients can run in the same machine of the server. For example, when I start firefox, firefox will start a X client that will communicate with the X server in the same machine to capture mouse and keyboard events but also make requests to the server to render the firefox window in the display.

Clients can run in a remote machine. That means that I can execute Firefox in a remote machine A, while a user interacts with its interface in the server machine. In this scenario, the server machine does not have any computation charge relative to the internals of Firefox execution, which is ran in the remote machine. But the application is controlled by some user with access to the mouse and keyboard of the machine where the X Server is running.

References

• The X Window System on wikipedia
• The history ox X11
• Xlib on wikipedia

2024-07-14

Threading study

Lock x RLock

RLock stands for reentrance lock. The only difference is that the acquire method can be called multiple times within the same thread without blocking. The release should be called the same amount of time to release the lock, that is, to allow a different thread to acquire it.

Lock x Event

Lock is used for preventing race conditions and preserve data integrity. It synchronizes the access, update of data or some logical sequence that cannot be executed simultaneously by different threads.

The main use case of locks is to synchronize execution within a same function by different threads.

def update(k):
    with lock:
        for e in random.sample(db,k):
            e.update()

with ThreadPoolExecutor(max_workers=5) as executor:
    executor.map(update,[2,1,4,5])

An Event is a communication mechanism that is more convenient to use between threads that are not executing the same function. An event is used by thread A executing function F to communicate to thread B executing function G an event, from which G should take a decision based on that.

event = threading.Event()

def F():
    event.set()

def G():
    while not event.is_set():
        time.sleep(0.1)
    do_something()

Condition

A condition is used when the same lock is needed to synchronize between different functions and different threads. It is a combination of a lock and an event. It is a lock because one needs to acquire a lock in order the condition turns true; and it is an event because one thread can signal another via the notify method that awakes thread that are waiting for the release of a lock.

def get_cake():
    while oven_available:
        while not cake_available():
            oven_available.wait() # releases oven_available lock and then waits notification, when 
                                  # an attempt to acquire the lock blocks until succeed.
        eat_cake()

def bake():
    while oven_available:
        bake_cake()
        cv.notify()

Here it is a simplified implementation of Condition in Python:

class Condition:
    def __init__(self):
        self._lock = threading.Lock()
        self._waiters = []

    def acquire(self):
        return self._lock.acquire()

    def release(self):
        return self._lock.release()

    def wait(self):
        # Release the lock, wait for notification, and reacquire the lock
        waiter = threading.current_thread()
        self._waiters.append(waiter)
        self.release()  # Release the lock
        threading._sleep(0)  # Release the GIL to allow other threads to acquire the lock
        while waiter in self._waiters:
            pass  # Busy wait until notified
        self.acquire()  # Reacquire the lock

    def notify(self, n=1):
        # Notify one or more waiting threads
        if len(self._waiters) > 0:
            for waiter in self._waiters[:n]:
                self._waiters.remove(waiter)

    def notify_all(self):
        # Notify all waiting threads
        self.notify(len(self._waiters))

2024-07-13

• Create tests for session.py.
• Refactor session.py
• Implement commands to quit and restart a session.