libra/docs/stories/story-8.10-admin-notification-system-events.md

Story 8.10: Admin Notification - System Events

Epic Reference

Epic 8: Email Notification System

User Story

As an admin, I want to be notified of critical system events, So that I can address issues promptly.

Dependencies

• Story 8.1: Email Infrastructure Setup (base template, SMTP configuration, queue setup)

Acceptance Criteria

Events to Notify

• Email delivery failures (SMTP errors, bounces)
• Scheduled job failures (Laravel scheduler)
• Queue job failures
• Critical application errors (database, payment, etc.)

Content Requirements

• Event type and description
• Timestamp (formatted per locale)
• Relevant details (error message, stack trace summary, job name)
• Recommended action (if applicable)
• Environment indicator (production/staging)

Delivery Requirements

• Sent immediately using sync mail driver (not queued)
• Clear subject line indicating urgency with event type
• Rate limited: max 1 notification per error type per 15 minutes
• Uses base email template from Story 8.1

Critical Error Definition

DO Notify for:

• Database connection failures (QueryException with connection errors)
• Queue connection failures
• Mail delivery failures (Swift_TransportException, mail exceptions)
• Scheduled command failures
• Any exception marked with ShouldNotifyAdmin interface

DO NOT Notify for:

• Validation exceptions (ValidationException)
• Authentication failures (AuthenticationException)
• Authorization failures (AuthorizationException)
• Model not found (ModelNotFoundException)
• HTTP 404 errors (NotFoundHttpException)
• CSRF token mismatches (TokenMismatchException)

Technical Notes

Files to Create

• app/Notifications/SystemErrorNotification.php
• app/Notifications/QueueFailureNotification.php
• app/Contracts/ShouldNotifyAdmin.php (marker interface)
• app/Services/AdminNotificationService.php (rate limiting logic)

Files to Modify

• bootstrap/app.php (exception handling)
• app/Providers/AppServiceProvider.php (queue failure listener)

Environment Variables

# Admin notification settings (add to .env.example)
ADMIN_NOTIFICATION_EMAIL=admin@libra.ps
SYSTEM_NOTIFICATIONS_ENABLED=true
ERROR_NOTIFICATION_THROTTLE_MINUTES=15

Configuration

// config/libra.php (add section)
'notifications' => [
    'admin_email' => env('ADMIN_NOTIFICATION_EMAIL'),
    'system_notifications_enabled' => env('SYSTEM_NOTIFICATIONS_ENABLED', true),
    'throttle_minutes' => env('ERROR_NOTIFICATION_THROTTLE_MINUTES', 15),
],

Admin Notification Service

// app/Services/AdminNotificationService.php
namespace App\Services;

use Illuminate\Support\Facades\Cache;
use Illuminate\Support\Facades\Notification;
use App\Notifications\SystemErrorNotification;

class AdminNotificationService
{
    public function notifyError(Throwable $exception): void
    {
        if (!config('libra.notifications.system_notifications_enabled')) {
            return;
        }

        $errorKey = $this->getErrorKey($exception);
        $throttleMinutes = config('libra.notifications.throttle_minutes', 15);

        // Rate limiting: skip if we've notified about this error type recently
        if (Cache::has($errorKey)) {
            return;
        }

        Cache::put($errorKey, true, now()->addMinutes($throttleMinutes));

        $adminEmail = config('libra.notifications.admin_email');

        // Send immediately (sync) - not queued
        Notification::route('mail', $adminEmail)
            ->notifyNow(new SystemErrorNotification($exception));
    }

    protected function getErrorKey(Throwable $exception): string
    {
        return 'admin_notified:' . md5(get_class($exception) . $exception->getMessage());
    }

    public function shouldNotifyAdmin(Throwable $exception): bool
    {
        // Check if exception implements marker interface
        if ($exception instanceof \App\Contracts\ShouldNotifyAdmin) {
            return true;
        }

        // Check against excluded exception types
        $excludedTypes = [
            \Illuminate\Validation\ValidationException::class,
            \Illuminate\Auth\AuthenticationException::class,
            \Illuminate\Auth\Access\AuthorizationException::class,
            \Illuminate\Database\Eloquent\ModelNotFoundException::class,
            \Symfony\Component\HttpKernel\Exception\NotFoundHttpException::class,
            \Illuminate\Session\TokenMismatchException::class,
        ];

        foreach ($excludedTypes as $type) {
            if ($exception instanceof $type) {
                return false;
            }
        }

        // Check for critical database errors
        if ($exception instanceof \Illuminate\Database\QueryException) {
            return $this->isCriticalDatabaseError($exception);
        }

        // Notify for mail exceptions
        if ($exception instanceof \Symfony\Component\Mailer\Exception\TransportExceptionInterface) {
            return true;
        }

        return false;
    }

    protected function isCriticalDatabaseError(\Illuminate\Database\QueryException $e): bool
    {
        $criticalCodes = ['2002', '1045', '1049']; // Connection refused, access denied, unknown DB
        return in_array($e->getCode(), $criticalCodes);
    }
}

Exception Handler Integration

// bootstrap/app.php
use App\Services\AdminNotificationService;

->withExceptions(function (Exceptions $exceptions) {
    $exceptions->reportable(function (Throwable $e) {
        $service = app(AdminNotificationService::class);

        if ($service->shouldNotifyAdmin($e)) {
            $service->notifyError($e);
        }
    });
})

System Error Notification

// app/Notifications/SystemErrorNotification.php
namespace App\Notifications;

use Illuminate\Notifications\Notification;
use Illuminate\Notifications\Messages\MailMessage;
use Throwable;

class SystemErrorNotification extends Notification
{
    public function __construct(
        public Throwable $exception
    ) {}

    public function via(object $notifiable): array
    {
        return ['mail'];
    }

    public function toMail(object $notifiable): MailMessage
    {
        $environment = app()->environment();
        $errorType = class_basename($this->exception);

        return (new MailMessage)
            ->subject("[URGENT] [{$environment}] System Error: {$errorType} - Libra Law Firm")
            ->greeting('System Alert')
            ->line('A critical error occurred on the platform.')
            ->line("**Error Type:** {$errorType}")
            ->line("**Message:** " . $this->exception->getMessage())
            ->line("**File:** " . $this->exception->getFile() . ':' . $this->exception->getLine())
            ->line("**Time:** " . now()->format('Y-m-d H:i:s'))
            ->line("**Environment:** {$environment}")
            ->line('Please check the application logs for full stack trace and details.')
            ->salutation('— Libra System Monitor');
    }
}

Queue Failure Notification

// app/Notifications/QueueFailureNotification.php
namespace App\Notifications;

use Illuminate\Notifications\Notification;
use Illuminate\Notifications\Messages\MailMessage;
use Illuminate\Queue\Events\JobFailed;

class QueueFailureNotification extends Notification
{
    public function __construct(
        public JobFailed $event
    ) {}

    public function via(object $notifiable): array
    {
        return ['mail'];
    }

    public function toMail(object $notifiable): MailMessage
    {
        $jobName = $this->event->job->resolveName();
        $environment = app()->environment();

        return (new MailMessage)
            ->subject("[URGENT] [{$environment}] Queue Job Failed: {$jobName} - Libra Law Firm")
            ->greeting('Queue Failure Alert')
            ->line('A queued job has failed.')
            ->line("**Job:** {$jobName}")
            ->line("**Queue:** " . $this->event->job->getQueue())
            ->line("**Error:** " . $this->event->exception->getMessage())
            ->line("**Time:** " . now()->format('Y-m-d H:i:s'))
            ->line("**Attempts:** " . $this->event->job->attempts())
            ->line('Please check the failed_jobs table and application logs for details.')
            ->salutation('— Libra System Monitor');
    }
}

Queue Failure Listener Registration

// app/Providers/AppServiceProvider.php boot() method
use Illuminate\Support\Facades\Queue;
use Illuminate\Queue\Events\JobFailed;
use Illuminate\Support\Facades\Notification;
use App\Notifications\QueueFailureNotification;

public function boot(): void
{
    Queue::failing(function (JobFailed $event) {
        if (!config('libra.notifications.system_notifications_enabled')) {
            return;
        }

        $adminEmail = config('libra.notifications.admin_email');

        Notification::route('mail', $adminEmail)
            ->notifyNow(new QueueFailureNotification($event));
    });
}

Marker Interface

// app/Contracts/ShouldNotifyAdmin.php
namespace App\Contracts;

/**
 * Marker interface for exceptions that should trigger admin notification.
 * Implement this interface on custom exceptions that require admin attention.
 */
interface ShouldNotifyAdmin
{
}

Testing Approach

Unit Tests

// tests/Unit/Services/AdminNotificationServiceTest.php
test('shouldNotifyAdmin returns false for validation exceptions', function () {
    $service = new AdminNotificationService();
    $exception = new \Illuminate\Validation\ValidationException(validator([], []));

    expect($service->shouldNotifyAdmin($exception))->toBeFalse();
});

test('shouldNotifyAdmin returns true for exceptions implementing ShouldNotifyAdmin', function () {
    $service = new AdminNotificationService();
    $exception = new class extends Exception implements \App\Contracts\ShouldNotifyAdmin {};

    expect($service->shouldNotifyAdmin($exception))->toBeTrue();
});

test('rate limiting prevents duplicate notifications', function () {
    $service = new AdminNotificationService();
    $exception = new RuntimeException('Test error');

    Notification::fake();

    $service->notifyError($exception);
    $service->notifyError($exception); // Should be throttled

    Notification::assertSentTimes(SystemErrorNotification::class, 1);
});

Feature Tests

// tests/Feature/Notifications/SystemErrorNotificationTest.php
test('critical database error triggers admin notification', function () {
    Notification::fake();

    // Simulate a critical database error being reported
    $exception = new \Illuminate\Database\QueryException(
        'mysql',
        'SELECT 1',
        [],
        new \Exception('Connection refused', 2002)
    );

    $service = app(AdminNotificationService::class);
    $service->notifyError($exception);

    Notification::assertSentOnDemand(
        SystemErrorNotification::class,
        function ($notification, $channels, $notifiable) {
            return $notifiable->routes['mail'] === config('libra.notifications.admin_email');
        }
    );
});

test('queue failure triggers admin notification', function () {
    Notification::fake();

    // Dispatch a job that will fail
    $job = new class implements ShouldQueue {
        use Dispatchable, InteractsWithQueue, Queueable;
        public function handle() { throw new Exception('Test failure'); }
    };

    // Process the job (will fail and trigger notification)
    // Note: This requires queue to be sync for testing

    Notification::assertSentOnDemand(QueueFailureNotification::class);
});

test('validation exception does not trigger admin notification', function () {
    Notification::fake();

    try {
        validator([], ['email' => 'required'])->validate();
    } catch (\Illuminate\Validation\ValidationException $e) {
        $service = app(AdminNotificationService::class);

        if ($service->shouldNotifyAdmin($e)) {
            $service->notifyError($e);
        }
    }

    Notification::assertNothingSent();
});

test('system error notification contains required information', function () {
    $exception = new RuntimeException('Test error message');
    $notification = new SystemErrorNotification($exception);

    $mail = $notification->toMail(new AnonymousNotifiable());

    expect($mail->subject)->toContain('[URGENT]');
    expect($mail->subject)->toContain('RuntimeException');
    expect((string) $mail->render())->toContain('Test error message');
});

Definition of Done

• AdminNotificationService created with rate limiting
• SystemErrorNotification created and styled
• QueueFailureNotification created and styled
• ShouldNotifyAdmin marker interface created
• Exception handler integration in bootstrap/app.php
• Queue failure listener in AppServiceProvider
• Environment variables documented in .env.example
• Configuration added to config/libra.php
• Notifications sent immediately (sync, not queued)
• Rate limiting prevents notification spam
• Excluded exceptions do not trigger notifications
• Unit tests for shouldNotifyAdmin() logic
• Feature tests for notification triggers
• All tests pass
• Code formatted with Pint

Estimation

Complexity: Medium | Effort: 4 hours

Notes

• Admin email should be configured in production before deployment
• Consider monitoring the failed_jobs table separately for recurring failures
• Rate limiting uses cache - ensure cache driver is configured properly
• In local development, set SYSTEM_NOTIFICATIONS_ENABLED=false to avoid noise