viktor/wrongmove
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

dump images using aiohttp and concurrently

Browse source
This commit is contained in:
Viktor Barzin 2025-05-17 22:42:37 +00:00
parent 01ac24b4b7
commit 68cc70bd11
No known key found for this signature in database
GPG key ID: 4056458DBDBF8863
2 changed files with 30 additions and 28 deletions
Download patch file Download diff file Expand all files Collapse all files

3
crawler/2_dump_detail.py
View file

@ -19,8 +19,7 @@ async def dump_detail(listing_paths: list[str]):
async def _dump_detail_for_listing(listing: Listing):
incremental = True
if incremental and not listing.path_detail_json().exists():
if listing.path_detail_json().exists():
return
# for listing in tqdm(filtered_listings):