{"title":"Self-Healing Test Pipelines","slug":"self-healing-tests","category":"guides","summary":"Build test pipelines that learn from failures and adapt automatically using Synapse memory.","audience":["human","llm"],"tags":["guide","testing","self-healing","ci"],"difficulty":"advanced","updated":"2026-06-27","word_count":260,"read_minutes":1,"lang":"en","translated":true,"requested_lang":"en","content_markdown":"\n# Self-Healing Test Pipelines\n\nTraditional test suites break when the UI changes. Self-healing tests use\nSynapse memory to learn from past failures and adapt — reducing flaky tests\nand maintenance burden.\n\n## Concept\n\n```\n┌─────────┐  fails   ┌──────────┐  store   ┌──────────┐\n│  Test   │ ───────▶ │  Synapse │ ───────▶ │ Memories │\n│  Run    │          │  Memory  │          │ (failures)│\n└─────────┘          └──────────┘          └──────────┘\n                           ▲                     │\n                           │   recall            │\n                           │  before next run    │\n                           └─────────────────────┘\n```\n\n1. Test runs\n2. If it fails, store the failure (what went wrong, why, how to fix)\n3. Next run: recall relevant failures before executing\n4. Apply known fixes automatically\n\n## Implementation\n\n### Step 1: Test Wrapper\n\nWrap each test with memory recall/store:\n\n```python\nimport requests\nfrom datetime import datetime\n\nURL = \"https://synapse.schaefer.zone\"\nMIND_KEY = \"mk_...\"\n\ndef self_healing_test(test_name, test_fn):\n    \"\"\"Decorator: wrap a test with self-healing memory.\"\"\"\n    def wrapper():\n        # 1. Recall past failures for this test\n        past_failures = requests.get(\n            f\"{URL}/memory/search?q={test_name}+failure\",\n            headers={\"Authorization\": f\"Bearer {MIND_KEY}\"}\n        ).json()\n        \n        # 2. Run test with failure context\n        try:\n            test_fn(known_failures=past_failures)\n        except Exception as e:\n            # 3. Store the failure\n            store_failure(test_name, e, traceback.format_exc())\n            raise\n    \n    return wrapper\n\ndef store_failure(test_name, error, traceback_str):\n    requests.post(f\"{URL}/memory\",\n        headers={\"Authorization\": f\"Bearer {MIND_KEY}\",\n                 \"Content-Type\": \"application/json\"},\n        json={\n            \"category\": \"mistake\",\n            \"key\": f\"test_failure_{test_name}_{datetime.now().isoformat()}\",\n            \"content\": f\"Test: {test_name}\\nError: {error}\\nTrace:\\n{traceback_str}\",\n            \"tags\": [\"test\", \"failure\", test_name],\n            \"priority\": \"high\"\n        })\n```\n\n### Step 2: Adaptive Test Logic\n\nInside the test, check for known failures and apply fixes:\n\n```python\n@self_healing_test\ndef test_login_page(browser, known_failures=None):\n    browser.goto(\"https://app.com/login\")\n    \n    # Check if we've seen this page change before\n    if known_failures and known_failures.get(\"results\"):\n        for failure in known_failures[\"results\"]:\n            if \"button moved\" in failure[\"content\"].lower():\n                # Use accessibility label instead of coordinates\n                browser.click(by_label=\"Login button\")\n                return\n    \n    # Default: use coordinates\n    browser.click(x=150, y=400)\n```\n\n### Step 3: Recovery Strategies\n\nStore recovery strategies as memories:\n\n```python\ndef store_recovery(failure_type, strategy):\n    requests.post(f\"{URL}/memory\",\n        headers={\"Authorization\": f\"Bearer {MIND_KEY}\",\n                 \"Content-Type\": \"application/json\"},\n        json={\n            \"category\": \"skill\",\n            \"key\": f\"recovery_{failure_type}\",\n            \"content\": strategy,\n            \"tags\": [\"test\", \"recovery\", failure_type],\n            \"priority\": \"high\"\n        })\n\n# Store recoveries for common failures\nstore_recovery(\"element_not_found\",\n    \"When element not found by ID, try by CSS class, then by XPath, \"\n    \"then by accessibility label. Take screenshot for debugging.\")\n\nstore_recovery(\"timeout\",\n    \"Increase timeout to 30s. If still fails, check if page is loading \"\n    \"dynamically — wait for specific element instead of fixed time.\")\n\nstore_recovery(\"stale_element\",\n    \"Re-find element before each interaction. Don't cache element references \"\n    \"across page transitions.\")\n```\n\n### Step 4: CI Integration\n\n```yaml\n# .gitlab-ci.yml\ntest:self-healing:\n  script:\n    - export SYNAPSE_MIND_KEY=$SYNAPSE_TEST_MIND_KEY\n    - pytest tests/ --self-healing\n  after_script:\n    # Summarize new failures\n    - python scripts/synapse_failure_summary.py\n```\n\n### Step 5: Failure Analysis Dashboard\n\n```python\n# Get all test failures from the last week\nr = requests.get(\n    f\"{URL}/memory/search?q=test+failure\",\n    headers={\"Authorization\": f\"Bearer {MIND_KEY}\"}\n)\n\n# Group by test name\nfailures = {}\nfor mem in r.json().get(\"results\", []):\n    test_name = extract_test_name(mem[\"content\"])\n    failures.setdefault(test_name, []).append(mem)\n\n# Report\nfor test, fails in sorted(failures.items(), key=lambda x: -len(x[1])):\n    print(f\"{test}: {len(fails)} failures\")\n```\n\n## Best Practices\n\n> [!TIP]\n> - **Store tracebacks** — they contain the exact line that failed\n> - **Tag by test name** — enables fast filtering\n> - **Use `mistake` category** — separates from regular memories\n> - **Set `high` priority** — failures should never be forgotten\n> - **Periodic cleanup** — delete memories for resolved issues\n> - **Don't store sensitive data** — credentials, PII\n\n## Common Failure Patterns to Store\n\n| Failure Type | What to Store |\n|--------------|---------------|\n| Element not found | Selector tried, page state, screenshot |\n| Timeout | Wait time, what was being waited for |\n| Assertion failed | Expected vs actual value |\n| Network error | URL, status code, response body |\n| Permission denied | Required permission, current user role |\n\n## Next Steps\n\n- [Automated iOS Testing](/docs/guides/automated-testing-ios)\n- [Memory Best Practices](/docs/guides/memory-best-practices)\n- [Error Recovery Cookbook](/docs/llm-cookbook/error-recovery)\n","content_html":"<h1>Self-Healing Test Pipelines</h1>\n<p>Traditional test suites break when the UI changes. Self-healing tests use\nSynapse memory to learn from past failures and adapt — reducing flaky tests\nand maintenance burden.</p>\n<h2>Concept</h2>\n<pre><code class=\"hljs language-plaintext\">┌─────────┐  fails   ┌──────────┐  store   ┌──────────┐\n│  Test   │ ───────▶ │  Synapse │ ───────▶ │ Memories │\n│  Run    │          │  Memory  │          │ (failures)│\n└─────────┘          └──────────┘          └──────────┘\n                           ▲                     │\n                           │   recall            │\n                           │  before next run    │\n                           └─────────────────────┘</code></pre><ol>\n<li>Test runs</li>\n<li>If it fails, store the failure (what went wrong, why, how to fix)</li>\n<li>Next run: recall relevant failures before executing</li>\n<li>Apply known fixes automatically</li>\n</ol>\n<h2>Implementation</h2>\n<h3>Step 1: Test Wrapper</h3>\n<p>Wrap each test with memory recall/store:</p>\n<pre><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import</span> requests\n<span class=\"hljs-keyword\">from</span> datetime <span class=\"hljs-keyword\">import</span> datetime\n\nURL = <span class=\"hljs-string\">&quot;https://synapse.schaefer.zone&quot;</span>\nMIND_KEY = <span class=\"hljs-string\">&quot;mk_...&quot;</span>\n\n<span class=\"hljs-keyword\">def</span> <span class=\"hljs-title function_\">self_healing_test</span>(<span class=\"hljs-params\">test_name, test_fn</span>):\n    <span class=\"hljs-string\">&quot;&quot;&quot;Decorator: wrap a test with self-healing memory.&quot;&quot;&quot;</span>\n    <span class=\"hljs-keyword\">def</span> <span class=\"hljs-title function_\">wrapper</span>():\n        <span class=\"hljs-comment\"># 1. Recall past failures for this test</span>\n        past_failures = requests.get(\n            <span class=\"hljs-string\">f&quot;<span class=\"hljs-subst\">{URL}</span>/memory/search?q=<span class=\"hljs-subst\">{test_name}</span>+failure&quot;</span>,\n            headers={<span class=\"hljs-string\">&quot;Authorization&quot;</span>: <span class=\"hljs-string\">f&quot;Bearer <span class=\"hljs-subst\">{MIND_KEY}</span>&quot;</span>}\n        ).json()\n        \n        <span class=\"hljs-comment\"># 2. Run test with failure context</span>\n        <span class=\"hljs-keyword\">try</span>:\n            test_fn(known_failures=past_failures)\n        <span class=\"hljs-keyword\">except</span> Exception <span class=\"hljs-keyword\">as</span> e:\n            <span class=\"hljs-comment\"># 3. Store the failure</span>\n            store_failure(test_name, e, traceback.format_exc())\n            <span class=\"hljs-keyword\">raise</span>\n    \n    <span class=\"hljs-keyword\">return</span> wrapper\n\n<span class=\"hljs-keyword\">def</span> <span class=\"hljs-title function_\">store_failure</span>(<span class=\"hljs-params\">test_name, error, traceback_str</span>):\n    requests.post(<span class=\"hljs-string\">f&quot;<span class=\"hljs-subst\">{URL}</span>/memory&quot;</span>,\n        headers={<span class=\"hljs-string\">&quot;Authorization&quot;</span>: <span class=\"hljs-string\">f&quot;Bearer <span class=\"hljs-subst\">{MIND_KEY}</span>&quot;</span>,\n                 <span class=\"hljs-string\">&quot;Content-Type&quot;</span>: <span class=\"hljs-string\">&quot;application/json&quot;</span>},\n        json={\n            <span class=\"hljs-string\">&quot;category&quot;</span>: <span class=\"hljs-string\">&quot;mistake&quot;</span>,\n            <span class=\"hljs-string\">&quot;key&quot;</span>: <span class=\"hljs-string\">f&quot;test_failure_<span class=\"hljs-subst\">{test_name}</span>_<span class=\"hljs-subst\">{datetime.now().isoformat()}</span>&quot;</span>,\n            <span class=\"hljs-string\">&quot;content&quot;</span>: <span class=\"hljs-string\">f&quot;Test: <span class=\"hljs-subst\">{test_name}</span>\\nError: <span class=\"hljs-subst\">{error}</span>\\nTrace:\\n<span class=\"hljs-subst\">{traceback_str}</span>&quot;</span>,\n            <span class=\"hljs-string\">&quot;tags&quot;</span>: [<span class=\"hljs-string\">&quot;test&quot;</span>, <span class=\"hljs-string\">&quot;failure&quot;</span>, test_name],\n            <span class=\"hljs-string\">&quot;priority&quot;</span>: <span class=\"hljs-string\">&quot;high&quot;</span>\n        })</code></pre><h3>Step 2: Adaptive Test Logic</h3>\n<p>Inside the test, check for known failures and apply fixes:</p>\n<pre><code class=\"hljs language-python\"><span class=\"hljs-meta\">@self_healing_test</span>\n<span class=\"hljs-keyword\">def</span> <span class=\"hljs-title function_\">test_login_page</span>(<span class=\"hljs-params\">browser, known_failures=<span class=\"hljs-literal\">None</span></span>):\n    browser.goto(<span class=\"hljs-string\">&quot;https://app.com/login&quot;</span>)\n    \n    <span class=\"hljs-comment\"># Check if we&#x27;ve seen this page change before</span>\n    <span class=\"hljs-keyword\">if</span> known_failures <span class=\"hljs-keyword\">and</span> known_failures.get(<span class=\"hljs-string\">&quot;results&quot;</span>):\n        <span class=\"hljs-keyword\">for</span> failure <span class=\"hljs-keyword\">in</span> known_failures[<span class=\"hljs-string\">&quot;results&quot;</span>]:\n            <span class=\"hljs-keyword\">if</span> <span class=\"hljs-string\">&quot;button moved&quot;</span> <span class=\"hljs-keyword\">in</span> failure[<span class=\"hljs-string\">&quot;content&quot;</span>].lower():\n                <span class=\"hljs-comment\"># Use accessibility label instead of coordinates</span>\n                browser.click(by_label=<span class=\"hljs-string\">&quot;Login button&quot;</span>)\n                <span class=\"hljs-keyword\">return</span>\n    \n    <span class=\"hljs-comment\"># Default: use coordinates</span>\n    browser.click(x=<span class=\"hljs-number\">150</span>, y=<span class=\"hljs-number\">400</span>)</code></pre><h3>Step 3: Recovery Strategies</h3>\n<p>Store recovery strategies as memories:</p>\n<pre><code class=\"hljs language-python\"><span class=\"hljs-keyword\">def</span> <span class=\"hljs-title function_\">store_recovery</span>(<span class=\"hljs-params\">failure_type, strategy</span>):\n    requests.post(<span class=\"hljs-string\">f&quot;<span class=\"hljs-subst\">{URL}</span>/memory&quot;</span>,\n        headers={<span class=\"hljs-string\">&quot;Authorization&quot;</span>: <span class=\"hljs-string\">f&quot;Bearer <span class=\"hljs-subst\">{MIND_KEY}</span>&quot;</span>,\n                 <span class=\"hljs-string\">&quot;Content-Type&quot;</span>: <span class=\"hljs-string\">&quot;application/json&quot;</span>},\n        json={\n            <span class=\"hljs-string\">&quot;category&quot;</span>: <span class=\"hljs-string\">&quot;skill&quot;</span>,\n            <span class=\"hljs-string\">&quot;key&quot;</span>: <span class=\"hljs-string\">f&quot;recovery_<span class=\"hljs-subst\">{failure_type}</span>&quot;</span>,\n            <span class=\"hljs-string\">&quot;content&quot;</span>: strategy,\n            <span class=\"hljs-string\">&quot;tags&quot;</span>: [<span class=\"hljs-string\">&quot;test&quot;</span>, <span class=\"hljs-string\">&quot;recovery&quot;</span>, failure_type],\n            <span class=\"hljs-string\">&quot;priority&quot;</span>: <span class=\"hljs-string\">&quot;high&quot;</span>\n        })\n\n<span class=\"hljs-comment\"># Store recoveries for common failures</span>\nstore_recovery(<span class=\"hljs-string\">&quot;element_not_found&quot;</span>,\n    <span class=\"hljs-string\">&quot;When element not found by ID, try by CSS class, then by XPath, &quot;</span>\n    <span class=\"hljs-string\">&quot;then by accessibility label. Take screenshot for debugging.&quot;</span>)\n\nstore_recovery(<span class=\"hljs-string\">&quot;timeout&quot;</span>,\n    <span class=\"hljs-string\">&quot;Increase timeout to 30s. If still fails, check if page is loading &quot;</span>\n    <span class=\"hljs-string\">&quot;dynamically — wait for specific element instead of fixed time.&quot;</span>)\n\nstore_recovery(<span class=\"hljs-string\">&quot;stale_element&quot;</span>,\n    <span class=\"hljs-string\">&quot;Re-find element before each interaction. Don&#x27;t cache element references &quot;</span>\n    <span class=\"hljs-string\">&quot;across page transitions.&quot;</span>)</code></pre><h3>Step 4: CI Integration</h3>\n<pre><code class=\"hljs language-yaml\"><span class=\"hljs-comment\"># .gitlab-ci.yml</span>\n<span class=\"hljs-attr\">test:self-healing:</span>\n  <span class=\"hljs-attr\">script:</span>\n    <span class=\"hljs-bullet\">-</span> <span class=\"hljs-string\">export</span> <span class=\"hljs-string\">SYNAPSE_MIND_KEY=$SYNAPSE_TEST_MIND_KEY</span>\n    <span class=\"hljs-bullet\">-</span> <span class=\"hljs-string\">pytest</span> <span class=\"hljs-string\">tests/</span> <span class=\"hljs-string\">--self-healing</span>\n  <span class=\"hljs-attr\">after_script:</span>\n    <span class=\"hljs-comment\"># Summarize new failures</span>\n    <span class=\"hljs-bullet\">-</span> <span class=\"hljs-string\">python</span> <span class=\"hljs-string\">scripts/synapse_failure_summary.py</span></code></pre><h3>Step 5: Failure Analysis Dashboard</h3>\n<pre><code class=\"hljs language-python\"><span class=\"hljs-comment\"># Get all test failures from the last week</span>\nr = requests.get(\n    <span class=\"hljs-string\">f&quot;<span class=\"hljs-subst\">{URL}</span>/memory/search?q=test+failure&quot;</span>,\n    headers={<span class=\"hljs-string\">&quot;Authorization&quot;</span>: <span class=\"hljs-string\">f&quot;Bearer <span class=\"hljs-subst\">{MIND_KEY}</span>&quot;</span>}\n)\n\n<span class=\"hljs-comment\"># Group by test name</span>\nfailures = {}\n<span class=\"hljs-keyword\">for</span> mem <span class=\"hljs-keyword\">in</span> r.json().get(<span class=\"hljs-string\">&quot;results&quot;</span>, []):\n    test_name = extract_test_name(mem[<span class=\"hljs-string\">&quot;content&quot;</span>])\n    failures.setdefault(test_name, []).append(mem)\n\n<span class=\"hljs-comment\"># Report</span>\n<span class=\"hljs-keyword\">for</span> test, fails <span class=\"hljs-keyword\">in</span> <span class=\"hljs-built_in\">sorted</span>(failures.items(), key=<span class=\"hljs-keyword\">lambda</span> x: -<span class=\"hljs-built_in\">len</span>(x[<span class=\"hljs-number\">1</span>])):\n    <span class=\"hljs-built_in\">print</span>(<span class=\"hljs-string\">f&quot;<span class=\"hljs-subst\">{test}</span>: <span class=\"hljs-subst\">{<span class=\"hljs-built_in\">len</span>(fails)}</span> failures&quot;</span>)</code></pre><h2>Best Practices</h2>\n<div class=\"callout callout-ok\"></div><h2>Common Failure Patterns to Store</h2>\n<table>\n<thead>\n<tr>\n<th>Failure Type</th>\n<th>What to Store</th>\n</tr>\n</thead>\n<tbody><tr>\n<td>Element not found</td>\n<td>Selector tried, page state, screenshot</td>\n</tr>\n<tr>\n<td>Timeout</td>\n<td>Wait time, what was being waited for</td>\n</tr>\n<tr>\n<td>Assertion failed</td>\n<td>Expected vs actual value</td>\n</tr>\n<tr>\n<td>Network error</td>\n<td>URL, status code, response body</td>\n</tr>\n<tr>\n<td>Permission denied</td>\n<td>Required permission, current user role</td>\n</tr>\n</tbody></table>\n<h2>Next Steps</h2>\n<ul>\n<li><a href=\"/docs/guides/automated-testing-ios\">Automated iOS Testing</a></li>\n<li><a href=\"/docs/guides/memory-best-practices\">Memory Best Practices</a></li>\n<li><a href=\"/docs/llm-cookbook/error-recovery\">Error Recovery Cookbook</a></li>\n</ul>\n","urls":{"html":"/docs/guides/self-healing-tests","text":"/docs/guides/self-healing-tests?format=text","json":"/docs/guides/self-healing-tests?format=json","llm":"/docs/guides/self-healing-tests?format=llm"},"translations_available":["en","zh","hi","es","fr","ar","pt","ru","ja","de","it","ko","nl","pl","tr","sv","vi","th","id","uk"]}