Skip to content

Journal

WON'T DO

  • Implement the web-scrapper version of english collins (but do not publish this one!).
  • Hook it up to word-detective.

TODO

2024-03-29

  • Go through commits and collect learning aspects.

2024-03-26

  • [word-guru] Add tests
  • [word-guru] Publish
  • [word-def-plugins-multilanguage-chatgpt] Update dependencies
  • [word-def-plugins-multilanguage-chatgpt] Publish
  • Fix dependencies path (instead of pointing to git repository, point to PyPI)
  • Publish word-def-plugin-english-collins

2024-03-22

  • Publish word-def
  • [word-def-plugin-multilanguage-chatgpt] Use monkeypatch to create a mock version to the api call in the test files.
  • [word-def] Create command to list available dictionaries for a language.
  • Make sure all the packages above accepts python 3.8 and above.

OIDC (OpenId Connect)

In order to enable OIDC in a publish package action on github, we need to give write permissions to token

myjob:
    permissions:
      id-token: write

2024-03-20

  • Add integration test for english-collins plugin.
  • Check documentation of english collins api
  • Check documentation of english collins word def plugin
  • Check documentation of chatgpt word def plugin
  • Check documentation of word-def

2024-03-15

  • Add tests to word-def-plugin-multilanguage-chatgpt
  • Add documentation on how to word-def: How to test plugins.
  • Add integration tests to chatgpt plugin.
  • Setup github actions for chatgpt plugin.
  • Add design document regarding the multilanguage runtime factory creation.

2024-03-13

  • Create word-def-plugin-multilanguage-chatgpt.
  • Create mechanism of adapter factory wrap at runtime for multilanguage.
  • Add more tests to word-def to take into account multilanguage plugins.

2024-03-09

  • Created configuration file mechanism for word-guru.
  • Add get-pos-tag command to cli.
  • The LSP servers in LazyVim are managed by Mason. If we go to ~/.local/share/nvim we can find the mason folder and also the lsp servers. For example, I need that to install the python-lsp-black plugin in the pylsp venv (this is located in the mason file hierarchy) in order to pylsp make use o black instead of autopep8. The goal is to align the formatting of pylsp with pre-commit.

2024-03-08

  • Created the word-guru project.
  • Included the dictionary module in the word-guru projects.
  • Created prompts for:
    • get-definition
    • get-synonyme
    • get-usage-examples
    • get-reverse-definition

2024-03-02

  • Figure it out a way to test the plugins since they are not part of the word-def package.
  • The Collins API do not offer usage examples.
  • Release v0.1.0 of word-def.
  • Set up tests and documentation for word-def-plugins-english-collins.
  • How to make tox command run all the test environments? I mean, pytest, format, typecheck...
    • envlist = py38,lint,typecheck
  • The entryContent field of the collins API can return XML responses instead of HTML. This could be interesting for parsing because it is easier to parse an XML.

2024-03-01

  • Learned about the difference between Sequence (more general interface) and List.
  • Update how to create a plugin document.
  • Add an entry in the design document talking about dependency issues between plugins and word-def package.
  • Update version mechanism.
  • Add test api method get_definition.
  • Fixed how-to-create-plugin documentation.
  • Update interfaces to use Sequence instead of List.
  • Add get-synonyme command to CLI.
  • Add get-usage-examples command to CLI.

2024-02-29

  • Wrote a test for the PluginRegister. This test creates a temporary folder T and copy a fake plugin file, contained in the input folder of the test, into T. Then we add T to the sys.path which allows Python to find the module. Everything is done programmatically.

2024-02-23

  • Add design entry in word-def project (Plugin architecture).
  • Add "How to create a plugin" documentation in word-def.
  • Add Getting started with word-def documentation.
  • Generate API documentation for word-def.
  • Fix type check errors.
  • Add public proxy to PluginRegister.
  • Refactor CLI of word-def.

2024-02-21

  • Write documentation specifying which exceptions are handled by the word-def mechanism.
  • Analysed the HTML response of collins API and decided to ignore some of the information returned. Preference is given to the def tag, but I should probably also print the pos tag.
  • Implement get_pos_tag for english-collins.
  • Save the HTML response for garden to use as a model of the HTML response.

2024-02-20

  • Remove dependency from word-def-plugin-core. Using word-def instead.

2024-02-19

  • Add mermaid diagrams in the documentation: sphinxcontrib-mermaid.Need to list in the extensions list of conf.py besides listing it as a dependency in the documentation tox environment.
  • Add design note regarding the decision about protocol.

2024-02-17

  • Update broken links and installation instructions.
  • Setup a publish package workflow.
  • Publish my first python package in PyPI (collins-dictionary-api-client)
  • Learn about ODCI authentication (the newest and easiest way to authenticate third parties such as GitHub to publish on PyPI). Docs

2024-02-16

  • Learn how to add links in the MD documents rendered by MyST Parser
  • Write documentation for collins-dictionary-api-client.
  • Registered API tests with the pytest mark api.
  • Create a unittest for the module cli.
  • Setup integration test for collins-dictionary-api-client on github.

2024-02-09

  • Learn that we need to indicate the flag --implicit-namespaces to sphinx-apidoc if we have implicit namespaces in our package. Otherwise modules are not linked correctly.
  • Update the pretty-format-json hook.
- id: pretty-format-json
  args:
    - "--autofix"
    - "--indent=4"
    - "--no-sort-keys"
  • Updated the collins-dictionary-api-client tests and documentation.
  • Tried to implement other methods in the plugin-core protocol such as get-synonyme but I could not find a reliable way to return a good response using the Collins API methods available. Moreover, I tried to check the documentation page of the Collins API but it was not available. I sent an email to my contact for the Collins API.
  • I am thinking to proceed without implementing other methods than get-definition.

2024-02-08

  • Learn how to setup a dependency from git using a particular branch:
"word-def-plugin-core @ git+https://github.com/danoan/word-def-plugin-core.git@dmartins/v0.1.0"
  • Learn how to specify the test files to be part of the coverage in pytest.
[tool.coverage.run]
include = [
    "*/danoan/word_def/plugins/modules/english_collins.py"
]
  • Shell glob is not working anymore in tox. I can't use the command I was used to to run doctests. The glob pattern gets enclosed with single quotes.
python -m doctest docs/*.md -o NORMALIZE_WHITESPACE

  • Learn about SimpleNamespace. It creates an object from a dictionary, in which the keys become attributes. Useful to create mock objects. I used to create a mock for the requests.Response object.

  • Learn that pytest.raises returns a type pytest.ExceptionInfo. To access the exception instance I need to type e.value.

2024-02-03

  • Setup plugin register mechanism.
  • Stick to a Adapter API protocol.
  • Setup an initial version of the word-def CLI.
  • Update repositories. I pushed the changes in all the three repositories to dmartins/v0.1.0 branch.
Danger

I may have face issues with package installation from marcao since the git packages are installed from the master branch.

2024-01-31

  • Created the word-def-plugin-core to hold common classes to plugins.
  • Passed some time figuring out an issue with .cache/pip. The package was being installed with a different name than the one it has and that was leading to import errors. The issue was that pip was using the previous name.

Use ABC or Protocols

I am thinking about the Adapter class that every plugin package must implement. The ABC approach is the standard design, but it comes with some pitfalls as strong coupling between classes, which is the strongest point of Protocols.

# ABC 
class BaseAdapter(ABC):
    @abstractmethod

# Protocols 
class Adapter():

2024-01-25

  • Create word-def-plugins-english-collins package. This is a simple namespace package that is used as a plugin. If someone wants to use the english-collins within word-def, it must install this package.
  • Start working on the plugin register mechanism.

2024-01-24

  • Created the word-def package. This package will group dictionary adapters (plugins)
  • Created a basic test with the collins english dictionary. It is now able to return a list of definitions of a word.
  • I am currently using the collins-dictionary-api directly. I must create the proper plugin for it and package it. The idea is to use the plugins namespace of the word_def package.
  • Using beautifulsoup4 to parse html.
  • Learned how to slect html tags from css class names.
  • Tried to configure a html formatter in nvim (htmlbeautifier) but it does not work very well.
  • Used PEP 508 strings to reference to a package located in my local computer (file://).

2024-01-17

  • Handle JSON response.
  • Collect JSON responses and write them in this journal as a reference.
  • Update test cases to also test JSON responses.
  • Update API and CLI method names. For example, get_search is now search.

2024-01-15

  • Collect XML responses and write them in this journal as a reference.
  • Update lazy.vim plugins and setup xml formatter.

2024-01-09

  • Setup pytest for collins-dictionary-api-client
  • I tried first to compare the xml outputs, but it turns out too complicated. The XML libraries output XML strings in arbitrary order. Tried some alternative ways to compare using xml.etree.ElementTree.canocalize but without success.
  • Finally, decided that in test_api.py we will primarily test if the API entrypoint is working.
  • Update actions to pass --secret-key and --entrypoint to pytest.
  • Discover that we cannot use github action secrets within reusable workflows. In the caller workflow we need to set secrets: inherit.

2024-01-04