viktor/wrongmove
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

bugfix - pages start from 1

Browse source
This commit is contained in:
Viktor Barzin 2025-06-22 13:04:10 +00:00
parent 03d24aff99
commit 9eae30b0d6
No known key found for this signature in database
GPG key ID: 4056458DBDBF8863
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
crawler/1_dump_listings.py
View file

@ -100,7 +100,8 @@ async def _fetch_listings_with_semaphore(
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
result = [] result = []
# we don't know how many pages we have but we stop as soon as there's no more # we don't know how many pages we have but we stop as soon as there's no more
for page_id in range(999): for page_id in range(1, 3):
# seems like all searches stop at 1500 entries (page_id * page_size)
async with semaphore: async with semaphore:
try: try:
listing_query_result = await listing_query( listing_query_result = await listing_query(