# Skill: Email Campaign System

Guidance for developing the self-hosted Laravel 11 email campaign system in this
repository. Read this before writing new features, fixing bugs, or changing sending
behavior. It captures the project's conventions, pitfalls, and worked state.

## Stack & environment (local WAMP box)

- **PHP 8.3.28** (WAMP), **Composer 2.10**, **MySQL 8.4.7** (root, no password,
  DB `email_campaign` already created), SQLite available for tests.
- **Pest is unusable** here (needs PHP 8.4) — use **PHPUnit** (the default skeleton
  test stack the prompt permits). Do not add Pest tests.
- Tests run on **in-memory SQLite** (`phpunit.xml` sets `DB_CONNECTION=sqlite`,
  `DB_DATABASE=:memory:`). Production target is MySQL.
- **No Redis** server/extension and no `predis` installed ⇒ `CACHE_STORE=database`
  in `.env` (Redis config kept for production). Do not use the Redis cache driver
  locally. `schedule:list` requires a cache store, hence database.
- Mail is configured to the **SMTP relay `mail.itocv.com:587` (STARTTLS)**,
  authenticating as `info@itocv.com` (see the Live sending setup section).

## How sending works (don't break this)

```
routes/console.php (every 2 min, campaign-dispatch, ->withoutOverlapping()->name())
  → CampaignDispatchService::dispatchNextBatch()
      → SendCampaignEmailJob (queued per recipient, sends synchronously)
          → CampaignMail (personalized HTML+plain, signed unsubscribe link)
          → success: recipient 'sent' + SendingStat increments sent_count
          → failure: recipient 'failed', error logged, NO count increment
```

- `bounces:check` runs every 15 min → `BounceHandlingService` (php-imap) →
  `SuppressionService::suppress()`.
- Volume is throttled by the warm-up / steady-state daily cap, never a hard wall.

## Receiver purpose / audience (which contacts get mailed)

Contacts have an optional `purpose` label (a string tag, e.g. `customers`,
`corporate-clients`, `leads`, `live-test`).

- **Upload**: the receiver upload form has a "Purpose (audience label)" field.
  `ContactImportService::import($file, $type, ?$purpose)` sets it on each contact.
- **Campaign creation** (`/campaigns/create`) lets you target an audience:
  - leave both empty → every active receiver (default);
  - pick a purpose → only receivers with that purpose;
  - pick specific emails (multi-select) → only those receivers (intersects with purpose if both set).
- Suppressed addresses are always excluded regardless of the filter.
- `Contact::distinctPurposes()` returns the sorted list of non-empty receiver
  purposes for the dropdown; `Contact::scopePurpose()` filters a query by purpose.
- `CampaignRequest` validates that at least one active receiver matches the chosen
  filter (`No active receiver contacts match the selected purpose.`).

Keep the `type` (sender/receiver) distinct from `purpose` — `type` describes the
contact's role, `purpose` is a marketing/audience grouping. Sender imports may also
carry a purpose but it is not used for targeting.

## Conventions (follow existing style)

- `declare(strict_types=1)` in every PHP file. PSR-12. Eloquent only (no raw queries
  unless required). Thin controllers — put logic in `app/Services`.
- New business logic belongs in a **service class** in `app/Services`.
- Models use typed properties and relationships; define status constants on models.
- Update `README.md` and this file when you add visible features or change behavior.
- Verify with tests (`php artisan test`) and style (`vendor/bin/pint`) after changes.

## Pitfalls (learned the hard way)

- **`SendingStat.date` has NO date cast** (intentional). SQLite stores `Y-m-d`
  as a string; queries must match exactly. Adding a `'date' => 'date'` cast inflated
  values to timestamps and broke `SendingStat::today()` — never re-add it.
- **Laravel 11 `Schedule::call()->withoutOverlapping()` requires `->name()`**
  on the task first, or it throws a `LogicException`. Keep the name present.
- **Blade: use `@{{ }}` to echo literal JS template-mustache braces** inside HTML
  views (e.g. a live-refresh snippet). A raw `{{ '{{ $name }}' }}` nested-echo caused
  a 500 on `/templates/create` until fixed.
- **MySQL 8.4 index-length error (1071)** on `users.email` unique is fixed by
  `Schema::defaultStringLength(191)` in `AppServiceProvider::boot()`. Don't remove it.
- **Path length**: the repo sits deep under `C:\wamp64\www\code\mamun-bhai\...`.
  Long command lines can hit Windows limits; prefer `workdir` over `cd`, and keep
  file copies to short temp paths (e.g. `C:\Users\...\Temp\opencode`) if ever needed.
- The warm-up **day-7 cap is configurable** via `config('campaign.steady_state_daily_cap')`
  (default 5000, overridable by `CAMPAIGN_STEADY_STATE_DAILY_CAP`).
- **Daily caps are now the `warmup_caps` array** in `config/campaign.php`
  (`CAMPAIGN_WARMUP_CAPS`, comma-separated): default `50,150,400,800,1500,3000,5000`
  (7-day ramp); a **single value is a flat cap from day 1** (e.g. `5000` = 5k on day 1);
  several values ramp, e.g. `5000,7500,10000` = 5k→10k over 3 sending days. Change live
  with `php artisan warmup:set 5000 7500 10000` or via `.env` + `config:cache`.
  `WarmupScheduleSeeder`/`warmup:set` sync the `warmup_schedules` table (deactivating rows
  outside the configured range). **The ramp advances only on actual sending days**
  (`SendingStat` rows with `sent_count > 0`), never on stale calendar dates.
- **Throughput math**: dispatch runs every 2 min = 720/day; with
  `CAMPAIGN_MAX_BATCH_SIZE=20` the ceiling is 14,400/day (5k needs ~7/run, 10k needs
  ~14/run). The cap and the batch ceiling are **independent gates** — both must allow the
  target volume. `php artisan warmup:show --target=10000` prints the exact math.
- **dotenv truncates passwords containing `#`** unquoted (e.g. `BDtest@#$2026` →
  reads only `BDtest@`), causing SMTP `535 Incorrect authentication data`. Always
  quote such values: `MAIL_PASSWORD="BDtest@#$2026"`. Verify with a dump of
  `config('mail.mailers.smtp.password')` length if auth fails.
- **`campaign_recipients.error_message` is a `text` column now** (was `string` 255).
  Long SMTP/Swift error messages (with `Str::limit(..., 255)` still appending `...`
  past 255) overflowed VARCHAR(255), which caused the catch block in
  `SendCampaignEmailJob` to throw and mask the real send error. Keep it `text`.
- **Custom headers belong on a `headers(): Headers` method, NOT in the `Envelope`.**
  Passing `headers:` to `new Envelope(...)` throws `Unknown named parameter $headers`
  at send time. `CampaignMail` returns a `Mailables\Headers` from its own `headers()`
  method with `X-Auto-Response-Suppress: OOF, AutoReply`,
  `List-Unsubscribe: <signedUrl>`, and `List-Unsubscribe-Post: List-Unsubscribe=One-Click`.
  **Do not re-add `Precedence: bulk`** — it is a known spam signal for Gmail and was
  deliberately removed.
- **`APP_URL` must match Apache's real serving path.** This box serves the app at
  `http://localhost/code/mamun-bhai/EmailCampaignSystem/public` (DocumentRoot is
  `${INSTALL_DIR}/www`, no vhost maps the app to a root). `APP_URL=http://localhost`
  made the signed confirm/unsubscribe links render as `http://localhost/unsubscribe/...`,
  which returned Apache **404 Not Found**. With `APP_URL` set to the subdirectory base,
  `url()->signedRoute(...)` emits working links. If the app is ever proxied to a real
  domain (`mail.itocv.com`), point `APP_URL` there instead and clear the config cache.
- **Gmail inbox vs spam** (opt-in email "Please confirm your subscription" landed in
  spam). The fix was: resolve the `{{ $confirmUrl }}` token so the confirmation link is
  a real signed URL (was literal), remove fake placeholder footer text (fake
  `privacy@example.com`/phone/address), fix malformed nested `<p><h2>` HTML, and add the
  `List-Unsubscribe`/`Precedence: bulk` headers.
- **Campaign delivers to spam while testing on localhost — this is mostly NOT a code
  fix.** Root causes: (1) no SPF/DKIM/DMARC on `itocv.com` → Gmail treats `info@itocv.com`
  as an unauthenticated sender; (2) `APP_URL=http://localhost/...` makes every
  unsubscribe/confirm link `http://localhost/...` (broken + HTTP = spam signal);
  (3) no sending reputation / warm-up for this identity/IP. Code-level reductions we
  applied (they help but won't overcome unauthenticated sending): removed
  `Precedence: bulk` header, removed the made-up `www.itocv.com` footer site-URL (now
  derived from public `APP_URL` host and omitted on localhost), removed contradictory
  "do not reply" footer copy, and cleaned up the generic template subject/body. To reach
  the inbox you must publish **SPF + DKIM + DMARC** for the sending domain's DNS, serve
  the app on a **public HTTPS URL** and set `APP_URL` to it (so links are valid, public,
  HTTPS), and **warm up** the sender reputation over days (the warm-up schedule already
  gates volume). Testing to Gmail from a local/residential IP through the relay will keep
  landing in spam until those are in place.

## Live sending setup

- The real SMTP relay for this box is **`mail.itocv.com`** on port **587** (STARTTLS,
  `MAIL_ENCRYPTION=tls`), authenticating as `info@itocv.com`. `.env` was updated from
  the earlier `127.0.0.1:25` (nothing listens locally on 25) to this relay.
- **Campaign sender must be a real/authenticated identity.** The live senders are
  registered via `SenderSeeder` / `sender:setup` with real addresses and SMTP creds
  (`James@alphareaction.com`, `Michael@alphareaction.com`, `Robert@alphareaction.com`,
  `david@itocv.com`, `joseph@i2cv.com`, `info@itocv.com`), each able to authenticate on
  `mail.itocv.com:587`. Pick one on the campaign; `SenderMailerService` sends through that
  sender's own mailbox.
- **Per-sender SMTP auth**: `Sender` rows carry `smtp_host/port/username/password/encryption`.
  `SenderMailerService::mailerFor()` builds a per-sender Symfony transport; falls back to
  `Mail::mailer('smtp')` (the `.env` account) when a sender has no credentials.
- **Dispatch grabs pending recipients across ALL running campaigns** (not just one).
  During a controlled live test, pause the other seeded running campaigns, clear
  `DB::table('jobs')`, re-dispatch only your target campaign, then run
  `php artisan queue:work --stop-when-empty --tries=1`.
- The pipeline was exercised end-to-end for 3 live Gmail test contacts (campaign id 6,
  purpose `live-test`); all three sent successfully and appear as `sent`.
- Double Opt-In confirmation: contacts now have a nullable `confirmed_at` (migration
  `2026_08_30_000003`). The `confirm` signed route (`/confirm/{contact}`,
  `ConfirmSubscriptionController`) validates the signature (403 invalid) and stamps
  `confirmed_at`. The opt-in template's `{{ $confirmUrl }}` resolves to that signed link,
  and `{{ $email }}`/`{{ $name }}` also resolve at send time.


## Configuration

- Business-time tuning lives in `config/campaign.php`: steady-state cap, warm-up caps array,
  batch sizes (min 1 / max 20), bounce mailbox.
- Env vars documented in `.env.example` (campaign cap, **`CAMPAIGN_WARMUP_CAPS`**, batch
  sizes, IMAP bounce mailbox username/password/folder, `CAMPAIGN_BOUNCE_KEEP_ON_SERVER`).
- Warm-up daily caps are seeded by `Database\Seeders\WarmupScheduleSeeder` from
  `config('campaign.warmup_caps')` (default 50, 150, 400, 800, 1500, 3000, steadyStateCap);
  `warmup:set` edits them live. Inspect with `warmup:show`.
- **UI settings now override config**: the `settings` table (`App\Models\Setting`) stores
  `max_batch_size`, `min_batch_size`, and `steady_state_daily_cap` at runtime. The
  **Settings page** (`/settings`, `SettingsController`) edits them in the UI and also
  edits the warm-up caps (deactivates out-of-range days, add/remove days). Because
  `CampaignDispatchService::batchSize()` reads `Setting::getInt('max_batch_size', config…)`,
  DB settings take precedence over `.env` — the `.env`/config file is only the fallback.

## Testing workflow

```powershell
php artisan test        # in-memory sqlite + Mail::fake()
php artisan test --filter=WarmupServiceTest   # single suite
vendor/bin/pint         # PSR-12 fixer
vendor/bin/pint --test  # check only
php artisan migrate:fresh --seed   # reset MySQL schema + re-seed (warm-up, templates, campaigns)
php artisan schedule:list          # confirm campaign-dispatch & bounces:check
php artisan route:list             # confirm web routes
php artisan warmup:show --target=10000   # warm-up report + throughput math
php artisan warmup:set 5000 7500 10000   # set daily caps live
php artisan warmup:reset --confirm       # clear sending stats (restart ramp at day 1)
```

## Seeders

`DatabaseSeeder` runs five seeders in order (order matters — the campaign depends on
senders, the template, and the receivers):

1. `WarmupScheduleSeeder` — daily caps from `config('campaign.warmup_caps')` (default
   7-day ramp 50→…→5,000; a single configured cap = flat from day 1).
2. `SenderSeeder` — registers the **6 real SMTP senders** (host `mail.itocv.com:587`,
   TLS, per-sender credentials; same data as `sender:setup`). Idempotent on `email`.
3. `EmailTemplateSeeder` — seeds the **`Welcome Newsletter`** live template (one generic,
   professional subject + HTML/plain body using `{{ $name }}` placeholders). Edit this to
   your real copy before the first send.
4. `ReceiverSeeder` — imports the **~1000 real receivers** from `receiver_email_list.txt`
   (one email per line, no names) as `type=receiver`, `purpose=newsletter`,
   `status=active`. Idempotent on `email`.
5. `CampaignSeeder` — creates the single **`Welcome Newsletter — Live Launch`** campaign
   bound to the `Welcome Newsletter` template, sent from `info@itocv.com`, with every real
   receiver as a **pending** recipient. It is seeded **`draft`** so nothing sends until you
   edit the body and start it. No fake `sending_stats` are seeded (a clean warm-up history).

Use `php artisan db:seed` to run them again, or run a single one with
`php artisan db:seed --class=EmailTemplateSeeder`.

## Reports & lists

Static reporting is collected under **Reports** (nav link) via `ReportController`:

- `reports.index` — overview tallies (subscriptions / unsubscriptions / spam) plus a
  table of every campaign with its pending/sent/failed/bounced counts, each linking to
  its report.
- `reports.subscriptions` — contacts with `confirmed_at` set (opt-in confirmation).
- `reports.unsubscribed` — contacts with `status='unsubscribed'`.
- `reports.spam` — rows from the `spam_reports` table.
- `reports.campaign` — per-campaign static report: recipient status counts, progress %,
  and among its recipients the subscribed/unsubscribed/spam tallies, plus the recipient
  table with a per-row subscribed/not-confirmed badge.

**Spam tracking**: added a `spam_reports` table (`contact_id` nullable, `email` unique,
`reported_at`). A signed `GET /spam/{contact}` (`SpamReportController@store`,
`spam.store`) records a complaint and calls `SuppressionService::suppress($email,
'manual')` so the address stops receiving mail; it is a GET (not POST) to match the
unsubscribe/confirm links and avoid CSRF issues from an email click. The spam list
queries `spam_reports`.

Note: unsubscribe and spam complaint links increment nothing on `sending_stats`; only
the dispatch job increments `sent_count` (and bounces increment via `BounceHandlingService`).

## Current state

All **11 build steps were completed and verified**:
1. Laravel 11 skeleton + `php-imap/php-imap` v6.
2. 8 migrations + models, 5 services, jobs/mailables, controllers, form requests,
   console commands, scheduler, and all Blade views.
3. Seeders: warm-up schedule, 15 legal-standard email templates, and 5 realistic
   example campaigns.
4. **Receiver `purpose` / audience targeting** on contacts + campaign creation
   (purpose filter and/or specific-email selection), with tests.
5. **28 tests / 86 assertions pass**; Pint clean; migrations + seed work on MySQL;
   all routes and scheduler tasks registered; all UI pages render 200.
6. 3 live-test receiver contacts present under purpose `live-test`:
   `moshiur.rah2016@gmail.com`, `moshiur.bro@gmail.com`, `moshiurbhai2@gmail.com`.
7. **Live send verified**: SMTP relay switched to `mail.itocv.com:587` (STARTTLS).
   Campaign id 6 (`Live Test — Welcome / Consent`) sent all 3 mails via the real
   queue pipeline (status `sent`, today's `sending_stats.sent_count` incremented).
   Two bugs fixed along the way: dotenv `#` password truncation, and
   `error_message` VARCHAR(255)→text.
8. **Deliverability fixes (Gmail spam → inbox)**: `CampaignMail` now resolves
   `{{ $confirmUrl }}`/`{{ $email }}`/`{{ $unsubscribeUrl }}`/`{{ $name }}` tokens,
   and sets `List-Unsubscribe`, `List-Unsubscribe-Post`, `Precedence: bulk`, and
   `X-Auto-Response-Suppress` headers via its `headers()` method (not `envelope()`).
   Templates no longer carry fake placeholder footers or malformed nested `<p><h2>`
   HTML. The corrected opt-in email rendered and sent OK via `mail.itocv.com:587`
   from `info@itocv.com`.
9. **Unsubscribe/confirm links fixed (404 → working)**. Links 404'd because
   `APP_URL=http://localhost` didn't match Apache's real serving path
   (`DocumentRoot = ${INSTALL_DIR}/www`, no vhost; app is served at
   `http://localhost/code/mamun-bhai/EmailCampaignSystem/public`). Fix: set
   `.env` `APP_URL` to that subdirectory base so `url()->signedRoute()` emits working
   links (HTTP 200 verified). Because a subdir `APP_URL` makes Laravel tests fail (the
   test kernel can't route the extra path prefix on `route()`-generated URLs), also set
   `<env name="APP_URL" value="http://localhost"/>` in `phpunit.xml` so the suite stays
   green (28 passed) while live emails get the subdir links. Corrected opt-in re-sent to
   `moshiurbhai2@gmail.com` with working confirm/unsubscribe links. Note: for recipients
   on OTHER machines these `localhost` links are unreachable — to make them universally
   valid the app must be deployed behind a public domain (e.g. `https://mail.itocv.com`),
   with `APP_URL` set to it. Worth doing before real live campaigns.

10. **Reports & lists**: a **Reports** section (`ReportController`) shows subscription,
    unsubscription, and spam lists plus per-campaign static reports. Spam complaints are
    tracked in a new `spam_reports` table via a signed `GET /spam/{contact}` endpoint
    (records the complaint and suppresses the address). 7 new `ReportTest` cases added;
    full suite is now **35 tests / 109 assertions**; Pint clean.

11. **Live-deployment prep (real data + docs)**:
    - Rewrote the seeders for production instead of the legal-demo data:
      `SenderSeeder` (6 real senders + SMTP creds), `ReceiverSeeder` (imports the ~1000
      real receivers from `receiver_email_list.txt`), `EmailTemplateSeeder` (now a single
      generic `Welcome Newsletter` template), `CampaignSeeder` (a single **draft**
      `Welcome Newsletter — Live Launch` campaign from `info@itocv.com` with all real
      receivers **pending**). No fake `sending_stats` are seeded; `WarmupScheduleSeeder` is
      kept for warm-up caps. Seeder verified end-to-end on a scratch SQLite DB
      (6 senders / 1001 receivers / 1 template / 1 campaign / 1001 pending recipients).
    - `DatabaseSeeder` order: Warmup → Senders → Templates → Receivers → Campaign.
    - Confirmed the `html_body` stored in templates is **raw HTML** rendered via
      `{!! $htmlBody !!}` with `{{ $name }}`/`{{ $email }}` resolved by
      `CampaignMail::personalize()` (a regex, not Blade) at send time.
    - **Campaign text-body verified live**: sent the `CampaignMail` for the
      `Welcome Newsletter` template from `info@itocv.com` to the 3 moshiur inboxes
      (`moshiur.bro@gmail.com`, `moshiur.rah2016@gmail.com`, `moshiurbhai2@gmail.com`) via
      `SenderMailerService` — all 3 sent OK through `mail.itocv.com:587`.
    - Test suite is now **45 tests / 133 assertions**, Pint clean. The whole test suite is
      kept for regression protection (tests depend only on `WarmupScheduleSeeder`, not on
      the removed demo data).
    - `README.md` and this file updated for live deployment (real senders/receivers, the
      draft launch campaign, per-sender SMTP auth, pre-send checklist).

12. **Configurable 5k→10k sending + warm-up report**:
    - Daily caps are now the configurable `warmup_caps` array (`config/campaign.php`,
      env `CAMPAIGN_WARMUP_CAPS`): default 7-day ramp to 5,000; single value = flat cap
      from day 1 (`5000` → 5k on day 1); multi-value ramps (`5000,7500,10000` → 10k).
    - The ramp advances on **actual sending days** (`SendingStat.sent_count > 0`), so a
      fresh identity never jumps days off stale test records. `WarmupService` exposes
      public `currentDayNumber()`, `maxDay()`, `activeSendingDays()`, `schedule()`.
    - New commands: **`warmup:show`** (schedule table, today's day/cap/sent/remaining,
      throughput math for `--target=5000|10000` — 720 runs/day at 2-min cadence, batch
      ceiling 20 ⇒ max 14,400/day; 5k needs ~7/run, 10k needs ~14/run) and **`warmup:set`**
      (edit caps live, no re-seed; deactivates out-of-range days). Plus `warmup:reset`
      (step 10-era, clears `SendingStat` to restart the ramp).
    - `DashboardController::index()` now calls `WarmupService::currentDayNumber()` instead
      of its own stale calendar-date math.
    - Suite grown to **53 tests / 158 assertions**, Pint clean. `WarmupServiceTest` covers
      flat-cap and multi-value-ramp configs; new `WarmupCommandTest` covers `warmup:show`
      and `warmup:set`.
    - Docs updated: `.env.example` (`CAMPAIGN_WARMUP_CAPS`), `README.md` (cap config,
      commands, throughput math, DNS/HTTPS deploy checklist with SPF/DKIM/DMARC records
      for `itocv.com` / `alphareaction.com` / `i2cv.com` → `115.187.18.117`).

13. **Client-facing operation panel (Settings + sending reports)**:
    - New **Settings page** (`/settings`, `SettingsController`, nav link): edits warm-up
      daily caps in the UI (add/remove days, enable/disable, flat or ramp schedule) plus
      max/min batch size and steady-state cap, with live "today cap / sent / remaining"
      and a throughput table (720 two-minute slots/day, ceiling = 720 × max_batch).
    - New `settings` table + `App\Models\Setting` (key/value with cache).
      `CampaignDispatchService::batchSize()` now reads batch sizes from `Setting` first,
      falling back to `config/campaign.php` (`.env`). UI edits take effect without
      touching `.env`.
    - New `target_count` on campaigns (nullable, editable at campaign creation, validated
      in `CampaignRequest`). Campaign show/report pages show progress toward the target
      (e.g. sent 2,000,000 / 10,000,000 with %).
    - **Dashboard overhauled** to give clients their daily operational picture: Sent Today,
      Sent Yesterday, Last 7 Days, All Time; Today's Cap / Remaining; Active Campaigns count
      and Pending Recipients; a last-7-days bar chart; active campaigns with progress bars.
    - **Reports page** now leads with a sending overview (Sent Today, Sent Yesterday,
      Pending across all campaigns, All Time) and the **per-campaign report** adds Sent
      Today / Sent Yesterday / Pending-remaining / ETA (remaining ÷ avg daily rate) plus a
      7-day daily-sending bar chart and target progress.
    - Suite grown to **60 tests / 183 assertions**, Pint clean (new
      `SettingsControllerTest`, expanded `DashboardTest` and `ReportTest`).

14. **Modern UI redesign (all pages)**:
    - New sidebar layout in `layouts/app.blade.php`: purple gradient sidebar with the
      brand, emoji icons per section, **active page highlighting** (`request()->routeIs(...)`),
      sticky full-height sidebar, responsive (collapses to icon-only under 640px).
    - Shared design system in the layout: CSS variables, `.stat-card`, `.grid-*`,
      `.bar-track`/`.bar-fill` progress bars, modern `.btn` variants, `.badge`, tables,
      `.page-head` header row with subtitle, empty states, focus rings, print styles.
    - Redesigned every view: Dashboard (stat cards + 7-day bar chart + active-campaign
      progress bars), Settings (status cards, editable warm-up days with add/remove,
      batch engine, volume capacity grid), Campaigns index/create/show (progress bars,
      target banner, status badges), Reports index + per-campaign report, engagement
      lists, Templates, and Upload Contacts. All pages share page-head + subtitle +
      grid cards for a consistent, professional look.

There are **no pending or blocked tasks**.