There are 5 different types of influence methods:

1. Rationalization (logic)
2. Assertion (authority)
3. Negotiating (middle ground)
4. Inspiring (vision)
5. Bridging (relationship)

Overuse has the opposite effect.

Most importantly, we should humbly pick the method that resonates with the other party over our modus operandum.

Happy hackin'

Did I: fulfill my intention; aspire to the person I want to be; have fun.

I heard Ellen Hendriksen moonlight on one of my favorite podcasts. She talked about a question I ask myself a lot - did I do a good job?

As a people pleaser I crave the external acknowledgment. I need to internalize it instead. Intentional. Aspirational. Fun.

Do it(things) for the sake of it and having fun with everything you do.

Enjoy the ride where It's the journey not the destination.

Happy hackin'

References

• The Savvy Psychologishologist - podcast

• How to know you did a good job - Ellen Hendriksen

In cloud systems it's a given that a server will go down and another will come up to take its place. Consistent hashing is a technique that minimizes the thundering herd of cache misses when the number of servers changes.

If you have a fixed set of servers n and use a naive modulo hash to route requests, about (n-1) / n keys get remapped on a topology change — getting worse as cluster size grows. Modulo hashing has no memory of where a key was. Every topology change is a complete re-derivation from scratch, so the old and new assignments are essentially independent. Two independent uniform distributions over different-sized sets overlap very little.

Consistent hashing puts each server on a ring. Each incoming request is hashed to a position and the closest clockwise server handles the request. This reduces the number of keys that need to be redistributed to 1 / n. It trades a big coordinated miss storm for a small, localized one where only the keys of the preceding neighbor's range are affected. The load is further balanced by adding virtual nodes to the ring. Consistent hashing doesn't solve the problem of hot keys, though. That could be handled by request coalescing, more v-nodes for hot keys or local application caches.

References

• Thundering Herd Problem

This article is part of the system design series where I am summarizing chapters from The System Design Interview: Volume 1 / Volume 2 amongst other related content

Clients are identified by user_id, Ip address or other modalities where server side rate limiters offer more control and not prone to manipulations over their client side variations.

RateLimiters have jurisdiction of the whole system and are usually placed in a centralized location like an API Gateway. They can filter by urls (Layer 7) or by IPs via IPTables (Layer 4).

Algorithms:

1. Token bucket: request per token and automatically refilled at a predetermined rate with max bucket size. Requires a bucket per limit and might be hard to tune with only two parameters(fill rate and bucket size) but memory efficient and allows burst traffic.

2. Leaking bucket: implemented as a queue fixed length queue which drops excess requests and are processed at a fixed rate. Good for stable processing rates however burst traffic could fill up the queue and delay recent requests. There are only two parameters available for tuning (queue size and number of workers processing variable length requests)

3. Fixed window counter: Timeline is split into fixed sized windows with a predefined threshold which if a request counter exceeds is dropped. Burst traffic at the falling/starting edge of consecutive windows could breach the max threshold

4. Sliding window log: timestamps of requests are logged. If the log size for a time window exceeds a predetermined threshold its dropped. Capable of accuracy at the expense of excessive memory for timestamps of all (even rejected) requests.

5. Sliding windows counter: A predefined threshold is configured. The current request count is calculated as a weighted percentage current window’s sub interval, incremented by weighted percentage of the previous window’s interval. Since this is an approximation, thresholds could be exceed for bursty traffic.

Rules are stored on disk as configurations that can be periodically synced or pushed on update to a centralized cache used by the rate limiter middleware. Use in a single distributed memory cache like redis to avoid synchronization issues across rate limiter applications. Batch operations with pipelining and leverage sorted data sets so increment, decrement, expire and query values to avoid race conditions.

User X-RateLimit-[ Remaining | Limit | Retry-After ] headers informing clients of their limits are or when they can try back after a HTTP 429 “Too many Requests” response.

Avoid being rate limited using client side cache, honor RateLimit headers and add sufficient backoff time for any retry logic.

This article is part of the system design series where I am summarizing chapters from The System Design Interview: Volume 1 / Volume 2 amongst other related content

1. The golden rule of leadership (treat employees with the same respect, empathy, and consideration as you'd like)

2. Build with people (people are the asset)

3. Succumb to the invisible sign (make others feel important)

4. Praise people to success (recognition)

5. The art of listening (do it more than talking)

6. Criticize the act not the person (sandwiched between two heavy layers of praise)

7. Be a follow-through person (do what you say)

8. Be enthusiastic (it's contagious; derived from Greek - 'God within')

9. Leaders set the pace

10. People support what they help create

11. Open-door policy

12. Help other people get what they want (to get what you want)

13. Don't compromise your principles

14. Instill a matter of pride into your people's work

15. Don't rest on your laurels

16. Encourage taking calculated risks (avoid discouraging, being too hard on failure)

17. Enjoy your work with humor

18. Nothing happens till somebody sells

19. Don't hide behind policy

20. Be a problem solver (distinguishing between real and imaginary)

21. Less stress (environment)

22. Develop people (promote from within)

23. Live by the golden rule on and off the job

References

• Mary Kay Ash

• Outliers blog/podcast

why not prototype our lives?

The Framework

The Odyssey Plan asks you to map out three radically different 5-year futures:

• Plan 1: Your current trajectory. What does life look like if you stay the course?

• Plan 2: What would you do if Plan 1 vanished tomorrow? The rug-pull scenario.

• Plan 3: What would you do if money and other people's opinions didn't matter?

The key constraint: these need to be genuinely different paths, not variations of the same thing. Being a Staff Engineer at Company A vs Company B isn't two different lives. Same life with different logos.

Why It Works

Engineers love prototyping. We spike solutions, run experiments, validate assumptions. The Odyssey Plan is basically prototyping applied to career decisions.

The three questions force you to surface things you might be avoiding:

1. Unfinished business: What's nagging at you? That side project you never shipped? The domain you always wanted to work in? The skill you keep meaning to learn?

2. Contingency planning: Most of us don't think about what happens if our current situation disappears. Layoffs happen. Companies pivot. Industries shift. What's your backup?

3. Permission to dream: Plan 3 is where it gets interesting. Removing money and social pressures reveals what you actually want versus what you think you should want.

The Dashboard

For each plan, you rate yourself on four metrics:

• Resources: Do you have the time, money, skills, and network to pull this off?

• Likability: How excited are you about this future?

• Confidence: Can you actually make this happen?

• Coherence: Does this align with your values and worldview?

It's like a health check for life plans. Low confidence but high likability? Maybe you need to prototype that path with smaller experiments. High resources but low likability? You might be optimizing for the wrong thing.

Applying It

I spent an evening sketching out my three plans. The exercise bought a few things to light:

Plan 1 was comfortable but had some gaps I'd been ignoring. Plan 2 forced me to think about what I'd actually do if my current role evaporated. Plan 3 was the most revealing. When you strip away external validation, what's left?

The goal isn't to pick one plan and commit. It's to have more ideas, because more ideas means better ideas. Some elements from Plan 3 might be worth prototyping in Plan 1. Some contingencies from Plan 2 might be worth preparing for now.

Life design is just applying the engineering experimentation and prototyping mindset to career decisions instead of code.

The authors put it well: designers don't think their way forward, they build their way forward.

Happy hackin'!

References:

• Stanford Life Design Lab

• Odyssey Planning Worksheet

Flask application’s typically auto-create db tables on start

So running flask db upgrade you’ll get:

INFO [alembic.env] No changes in schema detected.

To create the initial migration, override the applications existing database with a temporary memory-based one

DATABASE_URL=sqlite:/// flask db migrate

Happy hackin’!

References

• Flask-Migrate: Documentation

• Alembic: Cookbook - Building an Up to Date Database from Scratch

• Miguel Grinberg: How To Add Flask-Migrate To An Existing Project

The Rules

or:

• Returns the first operand when it's truthy

• Returns the second operand when the first is falsy

and:

• Returns the first operand when it's falsy

• Returns the second operand when the first is truthy

Python objects are either truthy or falsy, it doesn't matter that or and and don't return Booleans! Leverage this to short circuit on assignment for idiomatic code and avoid a gotcha

Happy Hackin’

References

• Stephen Gruppetta on X

• The Python Coding Stack: Do You Really Know How or And and Work in Python?

• Python Docs: Truth Value Testing

• Python Docs: Boolean Operations

• Python Gotchas: Mutable default Arguments

task_result = AsyncResult(task_id, app=celery_app)

works correctly if you have a result backend configured and tasks are defined with @app.task(bind=True). The result would be STARTED; SUCCESS; FAILURE; RETRY; REVOKED.

When staring tasks, I’d always get a PENDING status regardless of the queried task_id. The PENDING state is celery's default state for any task whose status is unknown or has no history.

The workaround: write the task_id to Redis (Celery's result backend) on task start, then only report status for task_id's that exist there. Also enable track_started=True by default, long-running tasks in chains show as PENDING until completion, which is pointless for monitoring.

def trigger_chain(task_chain):
    future = task_chain.apply_async()
    celery_app.backend.client.set(f"{future.id}", "started", ex=86400)

def get_task_result(task_id: str):
    try:
        task_result = AsyncResult(task_id, app=celery_app)
        if task_result.status == "PENDING":
            # Access the backend to check if the task key exists
            backend = celery_app.backend
            task_key, backend_client = (
                backend.get_key_for_task(task_id), gettattr(backend, "client", None
            )

            # Check if the key exists in Redis
            if backend_client and 
                if not ( backend_client.exists(key) or backend_client.exists(task_id)):
                    # Task doesn't exist in the result backend
                    return jsonify(
                        {
                            "error": "Task not found",
                            "task_id": task_id,
                            "message": "No task with this ID exists in the result backend",
                        }
                    ), 404
        # Valid task
        response = {
            "task_id": task_id,
            "status": task_result.status,
        }

Happy hackin’

References:

• Celery: Task States

• Task State Transitions

• Celery: track_task_started

Type 1 resilience is elastic. Under stress you bend and can return to your previous state. A non plastic deformation of a spring. Type 2 resilience is growth under pressure where you become stronger. It’s weight training at the gym. This is anti-fragility.

Everyone know’s what P-T-S-D is. However P-T-Growth is where you actually grow under pressure. This happens when you ‘Learn to fail or fail to learn’. You deliberately learn from your failures.

Happiness is a spectrum. It’s not binary. Some days are better than others which you’ve to push through.

Spiritual . Physical . Intellectual . Relationship . Emotional - (SPIRE) are the 5 dimensions of well being who’s tenets are the quality of our relationships tied in with gratitude and growth.

References

• Don't chase happiness. Become antifragile | Tal Ben-Shahar: Full Interview

• PTSD

The prefetch multiplier (default 4) is an optimization to reduce round trips to the broker (queue), to eagerly fetch tasks at the node per available worker. This happens at the consumers not at the workers.

With 4 workers:

• 16 tasks consumed

• 12 are waiting for an available worker

• 4 are being concurrently processed

If you have short lived tasks this isn’t a problem. However if your workloads’ execution time varies, all task could potentially convoy on the same node and dramatically reducing the throughput per worker. It’s recommended to use a prefetch of 1 or 2 to reduce the effect.

If you strictly have IO bound tasks you should use gevent. Using cooperative multitasking via a coroutine mechanism, you can effectively have 100s of workers per worker which will all execute concurrently.

Some considerations:

• Use connection pooling for database or http access to avoid herding

• monkey patch correctly

• if the workload isn’t IO bound, you will run into issues with the GIL.

Since python 3.14, free threaded python is officially supported (phase 2). However it’s not production ready but it brings thread level parallelism to applications.

Happy Hackin’

References:

• worker_pool

• worker_concurrency

• Concurrency vs Parallelism

• worker_prefetch_multiplier

• IO bound tasks

• Gevent

• GIL

• Python free threads

• Anthony Shaw’s talk on parallelism with python (sub-interpreters / free threads)

When consumers are configured with auto-commit, the queue offset will commit on message delivery. If the process dies before processing, the message is effectively lost.

Manual commit - at least once delivery

When manually committing messages which is done after processing occurs, if the process dies before committing, the message will be reprocessed by another consumer as the offset isn’t updated.

Always close your connections

Kafka consumers are run in a long running processes or maybe as a cronjob. The process connects to the configured cluster, topic and group. On exit, you need to explicitly close the connection. If not the topic/group will enter a rebalancing state where no messages can be read by existing consumers causing a delay in processing or even message loss.

Rebalancing

When a consumer enters/leaves a group, dies or partitions are reassigned.

Use a consumer group

It will load balance non overlapping messages across consumers, automatically assigns partitions and manage offsets. However there is chance of reprocessing of messages during rebalancing.

Exactly once delivery

Can be configured at the server level, but drastically reduces throughput.

Be idempotent

Duplicate messages will be the norm. Use the strategies below to avoid reprocessing messages.

• Unique constraints on application tables

• Unique constraints on separate message table

• Use the outbox pattern when adding to the DB + producing a message

• Redis deduplication with NX

• Produce Application level messages with a unique ID

Don’t manually assign partitions

This bypasses coordination across consumer group of a topic. The application has to manage conflicts.

Static members to avoid rebalancing

You can specify a unique group_instance_id of each instance to avoid rebalancing and its effects. Can be managed via k8s.

The Pitfalls

1. Non-atomic database writes and event publishing

Writing to a database and firing an event isn't atomic. If either the DB or message broker fails, you get data inconsistency.

Solutions:

• Resilient producer that retries until the event reaches Kafka

• CDC (Change Data Capture) with Debezium - captures DB changes via binlog and produces them as events automatically

2. Using Event Sourcing everywhere

Event sourcing stores events instead of entity state. You reconstruct current state by replaying events. Sounds cool but adds serious complexity:

• Need snapshots to avoid performance degradation

• Harder to create generic libraries (unlike CRUD ORMs)

• Only eventual consistency

Better approach: CRUD + CDC. Simple reads from the database, with CDC publishing changes for downstream materialized views. Get the benefits without the complexity.

3. No context propagation

Debugging distributed event flows is hard. Unlike HTTP chains, events scatter across topics and services.

# Add request context to event headers
event_headers = {
    'requestId': request_id,
    'userId': user_id
}

Automatically propagate these IDs through your event chain. Makes filtering logs and events trivial during incident investigation.

4. Large event payloads

Events over 5MB kill broker performance. Three remedies:

• Compression - Kafka and Pulsar support lz4, snappy. Broker-level compression beats application-level

• Chunking - Split payloads into chunks with metadata for reassembly

• Object store reference - Store payload in S3, pass URL in event

5. Duplicate event processing

Most brokers guarantee at-least-once delivery. Events can be processed multiple times.

# Use optimistic locking with revisionId
def process_event(event):
    revision_id = event['revisionId']
    # Read current version
    entity = db.get(event['entityId'])
    if entity.revision_id != revision_id:
        return  # Already processed

    # Update with new revision
    entity.update(revision_id=revision_id + 1)

For Kafka, use topic-partition-offset as unique transaction ID.

The Migration Strategy

Migrate gradually. Mix HTTP/RPC with event-driven patterns as needed. CDC is the sweet spot - ensures consistency without full event-sourcing complexity.

Context propagation (pitfall #3) is critical for operations. Fix that early.

Compression and transaction IDs are good defaults even if you don't hit pitfalls #4 and #5 yet.

Happy hackin'!

References:

• Wix Engineering: Event Driven Architecture pitfalls

• Debezium: CDC Kafka connector

• DDD Aggregates via CDC-CQRS Pipeline using Kafka & Debezium

• Greyhound: Wix messaging platform

• Kafka: Exactly once semantics

To avoid this in django, use:

1. select_related (for one-to-one or one-to-many queries)

2. prefetch_related (for many-to-many, many-to-one or generic-relations queries)

where it gets the related rows with the initial query, as the name indicates.

Always try and minimize the queries to a shared resource.

Happy hackin’

References:

• Article: django and the n plus one queries problem

• Django: QuerySet:prefetch_related

Alembic is SQLAlchemy's database migration framework—think change management for your relational database. It autogenerates scripts tracking your ORM model changes, letting you recreate your data model at any point in time.

Most apps run migrations on startup. I noticed my logs would mysteriously vanish right after the migration check.

The culprit? This innocent-looking snippet in the auto-generated env.py:

if config.config_file_name is not None:
    fileConfig(config.config_file_name)

That fileConfig defaults to disable_existing_loggers=True (thanks, backwards compatibility!), nuking any logging you previously configured.

The fix: Either edit the generated env.py or configure your logging after migrations run.

Happy hackin'

References:

• Stackoverflow article

• Alembic

• SQLAlchemy

• loggging.config.fileConfig

• Domain is what you trying to encode for which is owned by domain experts/specialist of the area

• The domain model is an abstraction represents the domain you’re trying to capture from the owners

• The domain model should be described and communicated within a ubiquitous language

• An analyst model is independently derived from the domain not considering software and design

• Layers of DDD: UI; Application; Domain; Infrastructure;

• Entities are components that have identities and are mutable

• Value Objects are immutable attributes or components that enrich entities and immutable

• Aggregates is a single unit of change that own several entities of a domain model

• An Aggregate root is an entity conduit for external requests policing change to a model

• Services provide functionally at the application level or domain level with Aggregates

• Factories are used to create complicated aggregates and tightly coupled to to the domain

• Repositories are an abstraction of storage, retrieval and reconstitution of entities, coupled with infrastructure

• Iterative and continuous Refactoring provide deeper insights, refine inconsistencies or missing components while avoiding a rigid design and improving to an incisive, deep model

• Make constraints (invariables), process (methods or services) and (composable) specifications explicit

• A Bounded Context is a logical frame allowing model evolution, isolation by module, team, code and DB schemas. CI fosters independent, pure and unified growth but doesn’t handle Inter-model relationships which should be well defined

• A Context Map show relationships between bounded contexts capture the whole unified domain model

• To reduce duplication, a Shared Kernel is a overlapping subset of the domain model shared by teams

• Establish a Customer/Supplier relationship between dependent teams with the former defining integration tests for the latter to improve speed

• Conformist are customers that work within a one-sided supplier relationship building on a model that’s given provided there is a rich interface that fulfill their needs.

• An Anticorruption layer is a proxy between our clients and a 3rd party (legacy) systems. It’s usually implemented with Facades that our clients interface with which internally use their own adapters with translators to facilitate inter system communication and maintain separation and purity

• A separate ways strategy is employed to split an application that has disparate bounded contexts

• An open host service wraps a external service as a set of services made available to several of our clients that use it. The protocol is open and avoids a translator per client. Esoteric clients can have a translator to the protocol

• Distillation separates the essential parts of a large domain from generic sub domains used is several places. The core distillation is iterative and should be the main focus. Sub domains can use off the shelf solutions, be outsourced, use existing models or an in house solution

References:

• http://www.infoq.com/minibooks/domain-driven-design-quickly

• https://domaindrivendesign.org/

• Domain Design Design Book

• Main image via Claude

This article is part of the system design series where I am summarizing chapters from The System Design Interview: Volume 1 / Volume 2 amongst other related content

Task Loss Prevention Between Web Process and Broker

• Enable broker confirmation: Configure confirm_publish on RabbitMQ to ensure tasks are actually committed to the broker before the delay operation completes

• Pass data references, not data values: Use S3 URLs or database IDs instead of passing large Python objects as task arguments to prevent gigantic tasks that can crash workers

• Implement database-sourced task recovery: Use Celery Beat with periodic tasks that check the database and re-queue missed tasks (e.g., verification emails) for automatic recovery

Task Loss Prevention Between Broker and Worker

• Set task_acks_late = True: Tasks remain on the broker until the worker acknowledges completion, enabling redelivery if workers crash

• Use transaction.on_commit(): Only queue tasks after database transactions commit to avoid race conditions where tasks execute before data is saved

• Make tasks idempotent: Use ORM methods like get_or_create() and update_or_create() so tasks can be safely retried multiple times

• Wrap tasks in transaction.atomic(): Ensure database changes can be rolled back if tasks are interrupted

Worker Reliability Configuration

• Set task_reject_on_worker_lost = True: Enable task redelivery even when workers die from memory errors or SIGKILL signals

• Handle all exceptions properly: Treat task exceptions with the same care as 500 errors in web views, using Celery's retry functionality for intermittent failures

• Use RabbitMQ over Redis/SQS: RabbitMQ's connection-based redelivery is more reliable than visibility timeout mechanisms

Deployment Safety

• Empty queues before changing task signatures: Ensure no old tasks remain when modifying function parameters

• Avoid ETA/countdown tasks beyond a few seconds: These live in worker memory and complicate deployments

• Use graceful shutdown (SIGTERM): Avoid SIGKILL during deploys to prevent unintended task dropping

Alternative Approaches

• Use dedicated workflow tools for complex orchestration: Consider Prefect, Temporal, or Airflow instead of Celery Canvas for complex workflows

• Implement proper monitoring and alerting: Set up observability tools specifically for Celery task execution

• Configure task time limits and expiration: Prevent clogged queues and outdated notifications

Celery Canvas Best Practices

Workflow Patterns Demonstrated

• Single Sequential Task (all_in_one): Processes all work in one task iterating through items sequentially - simplest but not parallel

• Parallel Tasks with Join: Queues multiple tasks simultaneously but demonstrates why you should never call result.get() within a task - causes RuntimeError

• Chord Pattern: Uses chord to run parallel tasks and execute a callback after all complete - recommended for fan-out/fan-in workflows

• Starmap for Parameter Mapping: Uses starmap to efficiently map function calls over parameter tuples

• Fine-grained Parallelism: Shows how to break work into smaller parallel tasks for maximum throughput

Performance and Scalability Insights

• Database-intensive work: Parallel task execution isn't always better - database query optimization often outperforms more concurrent tasks for DB-heavy operations

• Task granularity matters: Breaking work into smaller tasks enables better parallelism but creates more overhead

• Concurrency configuration: Use appropriate worker concurrency settings (example shows -c 8) based on your workload

Canvas Primitives Usage

• Chord: Best for fan-out/fan-in patterns where you need to collect results after parallel execution

• Starmap: Efficient for mapping a function over multiple parameter sets

• Group: For simple parallel execution without result collection

• Chain: For sequential task dependencies

Setup and Configuration Best Practices

• Use RabbitMQ as broker: The examples use RabbitMQ via Docker for reliable message delivery

• Environment isolation: Uses direnv for clean Python virtual environment management

• Mac-specific configuration: Sets OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES for multithreading compatibility

• Proper worker scaling: Configure concurrency based on workload characteristics

Performance Testing Approach

• Comparative benchmarking: The repository demonstrates multiple approaches to the same problem for performance comparison

• Real-world simulation: Uses a voter registration scenario with 10,000 records to test scalability

• Timing analysis: Encourages comparing timestamps to measure actual performance differences

Key Takeaway

The repository emphasizes that Canvas primitives should be used judiciously - parallel execution isn't always faster, especially for database-intensive operations. The choice between sequential, parallel, or Canvas-based approaches should be based on the specific nature of your workload and performance testing results.

References

• Mixing reliability with Celery for delicious async tasks with Flávio Juvenal (DjangoCon US '23)

• Celery canvas best practices

• Recommended celery settings

Celery links & best practices:

• Official Celery Configuration Docs

• Celery Reliability

• Celery best practices

• Celery in production - fixing bugs

• Celery eta tasks demystified

• 5 tips for writing production ready celery tasks

• Common celery issues on django projects

• Working with asynchronous celery tasks

• Celery / rabbitmq common issues

• Celery workers without heartbeat

Enter the git-filter-repo utility that’s based on the native git-filter-branch command. It does a lot of things but for my use case I only filtered for the folder I need moved.

Here’s what I did:

Prepare your source repository

# RE-Clone the source repository (local version will be altered)
git clone /path/to/source-repo source-repo-filtered 
cd source-repo-filtered 

# Extract only the directory you want (keeps full history)
git filter-repo --path directory-to-move/

# Optional: Remove the directory prefix if you want it at the root
git filter-repo --path-rename directory-to-move/:

filtering will remove all other git objects

Prepare the destination repository

# Clone or navigate to your destination repository
git clone /path/to/destination-repo destination-repo
cd destination-repo

# Add the local filtered repository as a remote
git remote add source-filtered /path/to/source-repo-filtered

# Fetch the filtered history
git fetch source-filtered

Merge into the destination repository

# Create a new branch for the merge
git checkout -b merge-directory

# Merge the filtered history (allows unrelated histories)
git merge source-filtered/main --allow-unrelated-histories

# If you want the directory in a specific location, you might need additional renaming
# For example, to put it in a subdirectory:
git filter-repo --path-rename :new-location/

review the branch before you merge

Clean up the destination repository

# Remove the temporary remote
git remote remove source-filtered

# Clean up the filtered repository clone
rm -rf /path/to/source-repo-filtered

# Clone the source repository again
git clone /path/to/source-repo source-repo-filtered 
cd source-repo-filtered 

# Delete a moved folder
git filter-repo --path directory-moved/ --invert-paths

# update the remote
git push --force

References:

• https://github.com/newren/git-filter-repo

• https://git-scm.com/docs/git-filter-branch

The effort you put in will compound and will leave you more time.

Given the improvements with AI text generation, it might seem like a fools errand. There are situations where you still type and you’ll need to key in the prompts at the very least.

How

Watch: Full Course - How to Type 3x Faster by Ali Abdaal. It’s where I got most of the tools to start me on my improvement (1) (2) journey

You need to make https://www.keybr.com/ and https://10fastfingers.com/typing-test/english part or you daily 15 minute routine and get faster at typing!

References:

• XKCD: Is It worth your time?

• Full Course - How to Type 3x Faster

• Ali Abdaal

With the following project layout:

|- main.py #from tools import tools. OK!
|- tools # module
   | __init__.py
   | tools.py
|- scripts
   | __init__.py
   | use_tools.py #from tools import tools **ModuleNotFoundError: No module named 'tools'
|- .venv #virtual env directory

main.py and scripts/use_tools.py both import from tools/tools.py

However only python scripts/use_tools.py results in a ModuleNotFoundError. Below are the search paths when executing the script.

sys.path = [
    '~/sandbox/courses/langchain/ice_breaker/scripts', #directory of input script
    '~/.asdf/installs/python/3.13.2/lib/python313.zip', #stdlib
    '~/.asdf/installs/python/3.13.2/lib/python3.13', #stdlib
    '~/.asdf/installs/python/3.13.2/lib/python3.13/lib-dynload', #stdlib
    '~/sandbox/courses/langchain/ice_breaker/.venv/lib/python3.13/site-packages' #venv
]

Python will search for the tools module in

1. current working directory of the source file i.e. scripts

2. check PYTHONPATH

3. lastly, it will check python installation specific directories and activated virtualenv paths for standard-libs and site-packages (always use one)

To fix the issue you can use PYTHONPATH environment variable to add additional directories to sys.path.

Here it’s being done in the virtual environment’s activate script

# .venv/bin/activate 

deactivate () {
    # reset old environment variables
    if [ -n "${_OLD_VIRTUAL_PYTHONPATH:-}" ] ; then
        PATH="${_OLD_VIRTUAL_PYTHONPATH:-}"
        export PATH
        unset _OLD_VIRTUAL_PYTHONPATH
    fi
}

_OLD_VIRTUAL_PYTHONPATH="$PYTHONPATH"
export PYTHONPATH="$PWD:$PYTHONPATH"

May your modules now be found.

Happy hackin’w

References

• The initialization of the sys.path module search path

• PYTHONPATH