archive 11 stale skills: move to skills/archived/ to match local layout

dot_claude/skills/archived/asyncpg-sqlalchemy-temp-table/SKILL.md

@@ -0,0 +1,118 @@
+---
+name: asyncpg-sqlalchemy-temp-table
+description: |
+  Fix for "relation does not exist" errors when using PostgreSQL COPY with temp tables
+  via raw asyncpg connections obtained from SQLAlchemy async sessions. Use when:
+  (1) CREATE TEMP TABLE ... ON COMMIT DROP followed by COPY fails with "relation does not exist",
+  (2) using session.connection().get_raw_connection() to access the asyncpg driver connection,
+  (3) temp tables disappear between sequential statements on the same raw connection.
+  Root cause: raw asyncpg connections obtained through SQLAlchemy operate in autocommit mode,
+  so each statement is its own transaction and ON COMMIT DROP takes effect immediately.
+author: Claude Code
+version: 1.0.0
+date: 2026-02-08
+---
+
+# asyncpg + SQLAlchemy: Temp Table Dropped Before COPY
+
+## Problem
+When using PostgreSQL's COPY protocol via raw asyncpg connections obtained from SQLAlchemy
+async sessions, `CREATE TEMP TABLE ... ON COMMIT DROP` causes the temp table to be dropped
+before subsequent statements (like `copy_records_to_table`) can use it.
+
+## Context / Trigger Conditions
+- Using SQLAlchemy's `AsyncSession` with `asyncpg` driver
+- Accessing the raw asyncpg connection via:
+  ```python
+  conn = await session.connection()
+  raw = await conn.get_raw_connection()
+  asyncpg_conn = raw.dbapi_connection.driver_connection
+  ```
+- Creating a temp table with `ON COMMIT DROP` and then running COPY on it
+- Error: `asyncpg.exceptions.UndefinedTableError: relation "_tmp_tablename" does not exist`
+
+## Root Cause
+When you extract the raw asyncpg connection from SQLAlchemy, the connection operates in
+**autocommit mode** at the asyncpg level. Each `await asyncpg_conn.execute(...)` call is
+its own implicit transaction. So:
+
+1. `CREATE TEMP TABLE _tmp_foo (...) ON COMMIT DROP` — creates the table, transaction
+   commits, table is dropped
+2. `asyncpg_conn.copy_records_to_table("_tmp_foo", ...)` — table doesn't exist anymore
+
+This happens even though SQLAlchemy's session thinks it has a transaction open — the raw
+driver connection bypasses SQLAlchemy's transaction management.
+
+## Solution
+Replace `ON COMMIT DROP` with manual lifecycle management:
+
+```python
+async def _copy_upsert(session, table_name, columns, records, conflict_target=None):
+    conn = await session.connection()
+    raw = await conn.get_raw_connection()
+    asyncpg_conn = raw.dbapi_connection.driver_connection
+
+    tmp = f"_tmp_{table_name}"
+    col_list = ", ".join(columns)
+
+    # Drop any leftover temp table, then create without ON COMMIT DROP
+    await asyncpg_conn.execute(f"DROP TABLE IF EXISTS {tmp}")
+    await asyncpg_conn.execute(
+        f"CREATE TEMP TABLE {tmp} (LIKE {table_name} INCLUDING DEFAULTS)"
+    )
+
+    # COPY rows into temp table
+    await asyncpg_conn.copy_records_to_table(tmp, records=records, columns=columns)
+
+    # Upsert from temp into real table
+    conflict = f"ON CONFLICT {conflict_target} DO NOTHING" if conflict_target else "ON CONFLICT DO NOTHING"
+    await asyncpg_conn.execute(
+        f"INSERT INTO {table_name} ({col_list}) SELECT {col_list} FROM {tmp} {conflict}"
+    )
+
+    # Clean up
+    await asyncpg_conn.execute(f"DROP TABLE IF EXISTS {tmp}")
+```
+
+**Alternative**: Wrap all raw statements in an explicit asyncpg transaction:
+```python
+async with asyncpg_conn.transaction():
+    await asyncpg_conn.execute(f"CREATE TEMP TABLE {tmp} (...) ON COMMIT DROP")
+    await asyncpg_conn.copy_records_to_table(tmp, ...)
+    await asyncpg_conn.execute(f"INSERT INTO ... SELECT ... FROM {tmp} ...")
+```
+Note: This may conflict with SQLAlchemy's own transaction if one is already open
+(asyncpg will use a savepoint in that case).
+
+## Verification
+Run the bulk insert and confirm no `UndefinedTableError`. Check that data appears
+in the target table.
+
+## Example
+Before (broken):
+```python
+await asyncpg_conn.execute(
+    f"CREATE TEMP TABLE {tmp} (LIKE {table} INCLUDING DEFAULTS) ON COMMIT DROP"
+)
+# This line fails: relation "_tmp_health_records" does not exist
+await asyncpg_conn.copy_records_to_table(tmp, records=rows, columns=cols)
+```
+
+After (working):
+```python
+await asyncpg_conn.execute(f"DROP TABLE IF EXISTS {tmp}")
+await asyncpg_conn.execute(f"CREATE TEMP TABLE {tmp} (LIKE {table} INCLUDING DEFAULTS)")
+await asyncpg_conn.copy_records_to_table(tmp, records=rows, columns=cols)
+await asyncpg_conn.execute(f"INSERT INTO {table} (...) SELECT ... FROM {tmp} ...")
+await asyncpg_conn.execute(f"DROP TABLE IF EXISTS {tmp}")
+```
+
+## Notes
+- This only affects raw asyncpg connections obtained through SQLAlchemy. If you use
+  asyncpg directly (without SQLAlchemy), you control the transaction yourself.
+- The `DROP TABLE IF EXISTS` before CREATE handles the case where a previous call
+  crashed mid-way and left the temp table behind.
+- Using `LIKE table INCLUDING DEFAULTS` copies the column types and defaults but not
+  indexes or constraints, which is ideal for a staging table.
+- Temp tables without `ON COMMIT DROP` persist for the duration of the session (connection),
+  not just the transaction. This is fine since we explicitly drop them.