Don't mutate your paginated Queryset

Django's Paginator works to batch your query items into pages. Under the hood it uses lazy slicing which is accomplished in sql by offset and limit.

Warning altering the data affects the limit/offset of proceeding paginated pages

data               = [1,2,3,4,5,6,7,8,9,10]
data_in_pages      = [(1,2),(3,4),(5,6),(7,8),(9,10)]
limit_offset_pages = [[0,2],[2,2],[4,2],[6,2],[8,2]] #zero index

for limit, offset in limit_offset_pages:
    sliver = data[limit:limit+offset]
    for index, item in enumerate(sliver, start=limit):
        if not item % 2: # delete even numbers
            del data[index]

As an alternative take the top slice of your mutated queryset to guarantee nothing will be missed.

while items:=query_set()[:batch_size]:
    process_items(items) #mutator

Never get high (mutate) on your own supply.

References

Django's Paginator
Postgresql's Offset/limit

Don't mutate your paginated Queryset

Comments

More from this blog

8 Simple reminders

Design a url shortner

No regrets

Design a distributed unique id generator

luck surface area

References

Command Palette

Comments

More from this blog

References