Blog#

Filtering In Pandas Dataframe

Pandas dataframe is like a small database, we can use it to inject some data and do some in-memory filtering without any external SQL. This post is much like a summary of this StackOverflow thread.

Git Cheat Sheet

This is not a complete Git cheat sheet for everyone, this is just a personal cheat sheet.

Using Python SQLAlchemy session in concurrent threads or tasks

SQLAlchemy DB session is not thread safe for both sync and async session. AsyncSession is only a thin proxy on top of a Session

The concurrency model for SQLAlchemy's Session and AsyncSession is therefore Session per thread, AsyncSession per task.

The best way to ensure this use is by using the standard context manager pattern locally within the top level Python function that is inside the thread or task, which will ensure the lifespan of the Session or AsyncSession is maintained within a local scope.

For applications that benefit from having a "global" Session where it's not an option to pass the Session object to specific functions and methods which require it, the scoped_session approach can provide for a "thread local" Session object; see the section Contextual/Thread-local Sessions for background. Within the asyncio context, the async_scoped_session object is the asyncio analogue for scoped_session, however is more challenging to configure as it requires a custom "context" function.

Creating Custom Python Request Auth Class

When you need to use a complicated, or a non-standard API authentication method, or your dev and prd environments don't use the same API authentication method, it might be better to create a Python requests auth method to reduce your work.

Creating Multiple Redis Instance Services On Windows

Even Salvatore Sanfilippo (creator of Redis) thinks it's a bad idea to use multiple DBs in Redis. So we can install as many Redis instances as the number of DBs we need. This post will show you how to create multiple Redis instance as Windows service on the same Windows server.

Setting Pwsh Invoke-WebRequest Proxy

Different than Windows Powershell, Powershell Core doesn't use the system proxy setting on Windows. This post will show you an one-line command to set Powershell Core web cmdlets proxy.

Using Gitlab integrated CICD for Python project on Windows

Gitlab ships with its own free CICD which works pretty well. This post will give you an example of the CICD file .gitlab-ci.yml for a Python project running on Gitlab Windows runner.

Migrate Gitlab in docker

This post will walk you through the steps to migrate Gitlab from one docker container to another. The steps need you to know how to install a new Gitlab container and how to backup and restore Gitlab container, because the migration is just a restoration of a backup to another container.

Update Gitlab in docker

Gitlab has several methods to update to newer version depending on the type of the original installation and the Gitlab version. This post will show you the way for docker version of Gitlab, which is the simplest among others.

Terminate Powershell script or session

I always asked myself how to terminate a Powershell script or session, each time I needed to do some tests by myself and also searched on Google. But I could never remember it. So I would like to take this post to note it down, the next time I need to terminate, just need to come back to here.