Anomaly Detection

Sentinel includes a statistical anomaly detection system that identifies unusual traffic patterns by comparing real-time user activity against learned behavioral baselines. Unlike the WAF, which matches known attack signatures, anomaly detection catches previously unseen threats by flagging activity that deviates from what is normal for each user.

The detector builds per-user baselines from historical activity data — typical active hours, frequently accessed routes, source IPs, geographic locations, and request velocity. When new activity diverges significantly from these baselines, Sentinel emits a threat event with type AnomalyDetected.

Requires a UserExtractor

Anomaly detection operates on a per-user basis. You must configure a UserExtractor in your Sentinel config so the system can associate requests with user identities. Without it, the anomaly detector has no user context and will not run.

Configuration

Anomaly detection is disabled by default. Enable it by setting Enabled: true in your AnomalyConfig. The minimal configuration uses all defaults:

main.gogo
1package main
2

3import (
4    "time"
5

6    sentinel "github.com/MUKE-coder/sentinel"
7    "github.com/gin-gonic/gin"
8)
9

10func main() {
11    r := gin.Default()
12

13    sentinel.Mount(r, nil, sentinel.Config{
14        Anomaly: sentinel.AnomalyConfig{
15            Enabled:    true,
16            Sensitivity: sentinel.AnomalySensitivityMedium,
17        },
18        UserExtractor: func(c *gin.Context) *sentinel.UserContext {
19            return &sentinel.UserContext{
20                ID:    c.GetHeader("X-User-ID"),
21                Email: c.GetHeader("X-User-Email"),
22                Role:  c.GetHeader("X-User-Role"),
23            }
24        },
25    })
26

27    r.GET("/api/data", func(c *gin.Context) {
28        c.JSON(200, gin.H{"status": "ok"})
29    })
30

31    r.Run(":8080")
32}

AnomalyConfig Reference

FieldTypeDefaultDescription
EnabledboolfalseEnables anomaly detection. When false, the detector is a no-op.
SensitivityAnomalySensitivityAnomalySensitivityMediumControls the scoring threshold that triggers an anomaly event. See sensitivity levels below.
LearningPeriodtime.Duration7 * 24 * time.HourHow far back to look when computing a user's behavioral baseline. Longer periods produce more stable baselines but are slower to adapt.
Checks[]AnomalyCheckTypeAll checks enabledWhich anomaly checks to run. If empty, all check types are enabled. Specify a subset to narrow detection scope.
config.gogo
1Anomaly: sentinel.AnomalyConfig{
2    Enabled:        true,
3    Sensitivity:    sentinel.AnomalySensitivityMedium,
4    LearningPeriod: 14 * 24 * time.Hour, // 2 weeks of history
5    Checks: []sentinel.AnomalyCheckType{
6        sentinel.CheckOffHoursAccess,
7        sentinel.CheckUnusualAccess,
8        sentinel.CheckVelocityAnomaly,
9        sentinel.CheckImpossibleTravel,
10        sentinel.CheckDataExfiltration,
11    },
12}

Sensitivity Levels

Sensitivity controls the anomaly score threshold required to emit a threat event. Each anomaly check contributes a score, and the total is compared against the threshold. Lower thresholds trigger more alerts.

LevelConstantThresholdBehavior
Lowsentinel.AnomalySensitivityLow50Only the most significant anomalies trigger events. Fewer alerts, less noise. Best for high-traffic applications where some deviation is normal.
Mediumsentinel.AnomalySensitivityMedium30Balanced detection. Catches meaningful deviations without overwhelming you with alerts. Recommended starting point.
Highsentinel.AnomalySensitivityHigh15Aggressive detection. Catches subtle anomalies but produces more alerts. Best for high-security environments or during active incident investigation.

Recommended Starting Point

Start with AnomalySensitivityMedium. Monitor the anomaly events in the dashboard for a week to understand your application's normal patterns, then adjust up or down based on the signal-to-noise ratio you observe.

What Gets Detected

The anomaly detector runs a configurable set of behavioral checks against each user activity event. Each check compares one aspect of the current request against the user's baseline and returns a score from 0 (normal) to 30 (highly anomalous). Scores from all checks are summed (capped at 100) and compared against the sensitivity threshold.

CheckConstantMax ScoreWhat It Detects
Off-Hours AccessCheckOffHoursAccess30Activity during hours when the user is rarely active. If the current hour represents less than 1% of baseline activity, the full score is assigned; less than 3% yields a partial score of 15.
Unusual Route AccessCheckUnusualAccess25Access to routes (method + path) that the user has never accessed before during the learning period. Useful for detecting lateral movement or account compromise.
Velocity AnomalyCheckVelocityAnomaly25Request rate spikes that exceed 3x the user's average rate for the current time of day. Catches automated scraping, credential stuffing, or bot activity on a compromised account.
Impossible TravelCheckImpossibleTravel30Activity from a new IP address, especially from a country the user has never connected from. A new country yields a score of 30; a new IP in the same country yields 10.
Data ExfiltrationCheckDataExfiltration20Unusually large response sizes or durations that exceed 5x the user's baseline average. Indicates potential bulk data extraction.
Credential StuffingCheckCredentialStuffing--Reserved check type for detecting credential stuffing patterns. Configure via Auth Shield for full brute-force protection.
// Enable only specific checks
Checks: []sentinel.AnomalyCheckType{
    sentinel.CheckOffHoursAccess,
    sentinel.CheckImpossibleTravel,
    // Omit checks that are not relevant to your application
}

Minimum Baseline Requirement

The anomaly detector requires at least 10 historical activity records for a user before it will evaluate checks. Users with fewer records are silently skipped. This prevents false positives on new users or accounts with very little history.

How It Works

The anomaly detector operates as a handler in Sentinel's asynchronous event pipeline. It does not sit in the HTTP request path and adds no latency to your responses.

  1. Activity Tracking — When a request arrives, Sentinel records a UserActivity event containing the user ID, timestamp, IP address, HTTP method, path, response duration, and geographic country (if geo is enabled).
  2. Pipeline Dispatch — The activity event is sent to the async pipeline via a non-blocking ring buffer. This decouples detection from request handling.
  3. Baseline Computation — The detector loads the user's historical activity from storage (within the configured LearningPeriod) and computes a UserBaseline. Baselines are cached for 1 hour to avoid repeated computation.
  4. Check Evaluation — Each enabled check compares the current activity against the baseline and returns a score. Scores are summed and capped at 100.
  5. Threshold Comparison — If the total score meets or exceeds the sensitivity threshold, a ThreatEvent is emitted with type AnomalyDetected.
  6. Event Processing — The threat event flows through the rest of the pipeline: it is persisted to storage, updates the threat actor profile, recalculates the security score, and triggers alerts if the severity meets the alerting threshold.
# Anomaly detection flow:
#
# HTTP Request ──> Record UserActivity ──> Async Pipeline
#       │                                       │
#       v                                       v
#   Response sent                    Load/Compute Baseline
#   (no added latency)                          │
#                                               v
#                                    Run Checks (score each)
#                                               │
#                                    ┌──────────┴──────────┐
#                                    v                      v
#                              Score < Threshold      Score >= Threshold
#                                    │                      │
#                                    v                      v
#                                No action           Emit ThreatEvent
#                                                          │
#                                                          v
#                                                  Persist + Alert

Non-Blocking Detection

The anomaly detector runs entirely outside the HTTP request path. Activity recording uses a non-blocking ring buffer, so detection never slows down your API responses, even when computing baselines from large activity histories.

Baselines

A UserBaseline captures the behavioral profile for a single user. It is computed from all activity within the LearningPeriod and cached in memory for 1 hour before being recomputed.

Baseline FieldWhat It TracksUsed By
ActiveHoursDistribution of activity across hours of the day (0-23)CheckOffHoursAccess
TypicalRoutesSet of method+path combinations the user has accessedCheckUnusualAccess
AvgRequestsPerHourAverage request rate across the learning periodCheckVelocityAnomaly
SourceIPsSet of IP addresses the user has connected fromCheckImpossibleTravel
CountriesSet of countries (by geo lookup) the user has connected fromCheckImpossibleTravel
AvgResponseSizeAverage response duration/size across the learning periodCheckDataExfiltration

The LearningPeriod determines the time window for baseline computation. The default is 7 days. A longer period (e.g., 14 or 30 days) produces more stable baselines but adapts more slowly to legitimate changes in user behavior.

// Short learning period — adapts quickly, less stable
LearningPeriod: 3 * 24 * time.Hour,  // 3 days


// Default — balanced
LearningPeriod: 7 * 24 * time.Hour,  // 7 days


// Long learning period — very stable, slow to adapt
LearningPeriod: 30 * 24 * time.Hour, // 30 days

Anomaly Events

When an anomaly is detected, Sentinel emits a ThreatEvent with the threat type AnomalyDetected. These events appear in the dashboard alongside WAF detections and other security events.

Event FieldValue
ThreatTypes["AnomalyDetected"]
SeverityComputed from the anomaly score: Critical (≥80), High (≥60), Medium (≥30), Low (<30)
ConfidenceThe raw anomaly score (0-100), representing how far the activity deviates from baseline
EvidenceContains the anomaly score and the list of checks that contributed to it
Blockedfalse — anomaly events are informational; they do not block requests

Severity Mapping

The anomaly score is mapped to a severity level using the following thresholds:

Score RangeSeverityInterpretation
80 - 100CriticalMultiple strong anomaly signals. Very likely a compromised account or active attack.
60 - 79HighSignificant deviation from baseline. Warrants immediate investigation.
30 - 59MediumModerate deviation. Could be a legitimate change in behavior or early sign of compromise.
0 - 29LowMinor deviation. Usually benign but logged for audit purposes.

Anomaly Events Are Non-Blocking

Anomaly events are always informational — they never block requests. The detector flags suspicious behavior so you can investigate, but it does not interrupt user sessions. To automatically respond to anomalies, pair anomaly detection with the alerting system to receive notifications when high-severity anomalies occur.

Full Configuration Example

Below is a complete example that enables anomaly detection alongside other Sentinel features:

main.gogo
1package main
2

3import (
4    "time"
5

6    sentinel "github.com/MUKE-coder/sentinel"
7    "github.com/gin-gonic/gin"
8)
9

10func main() {
11    r := gin.Default()
12

13    sentinel.Mount(r, nil, sentinel.Config{
14        // Enable anomaly detection
15        Anomaly: sentinel.AnomalyConfig{
16            Enabled:        true,
17            Sensitivity:    sentinel.AnomalySensitivityMedium,
18            LearningPeriod: 7 * 24 * time.Hour,
19            Checks: []sentinel.AnomalyCheckType{
20                sentinel.CheckOffHoursAccess,
21                sentinel.CheckUnusualAccess,
22                sentinel.CheckVelocityAnomaly,
23                sentinel.CheckImpossibleTravel,
24                sentinel.CheckDataExfiltration,
25            },
26        },
27

28        // Required: tell Sentinel how to identify users
29        UserExtractor: func(c *gin.Context) *sentinel.UserContext {
30            userID := c.GetHeader("X-User-ID")
31            if userID == "" {
32                return nil // Unauthenticated request
33            }
34            return &sentinel.UserContext{
35                ID:    userID,
36                Email: c.GetHeader("X-User-Email"),
37                Role:  c.GetHeader("X-User-Role"),
38            }
39        },
40

41        // Enable geo for country-based anomaly checks
42        Geo: sentinel.GeoConfig{
43            Enabled: true,
44        },
45

46        // Alert on high-severity anomalies
47        Alerts: sentinel.AlertConfig{
48            MinSeverity: sentinel.SeverityHigh,
49            Webhook: &sentinel.WebhookConfig{
50                URL: "https://hooks.example.com/sentinel",
51            },
52        },
53    })
54

55    r.GET("/api/data", func(c *gin.Context) {
56        c.JSON(200, gin.H{"status": "ok"})
57    })
58

59    r.Run(":8080")
60}

Testing Considerations

Anomaly detection requires sufficient historical data to produce meaningful baselines. This makes it harder to test than the WAF or rate limiter, which respond immediately. Here are strategies for testing effectively.

Building a Baseline

The detector requires at least 10 activity records for a user before it will evaluate checks. In a test environment, you can seed activity data by sending authenticated requests over a consistent pattern, then introducing an anomalous request to verify detection.

# Seed baseline activity (repeat over multiple hours/days for a realistic baseline)
for i in $(seq 1 20); do
    curl -s -H "X-User-ID: testuser" http://localhost:8080/api/data > /dev/null
    sleep 1
done


# Now trigger an anomaly — access from a different IP or unusual route
curl -s -H "X-User-ID: testuser" http://localhost:8080/admin/settings
# Check the Sentinel dashboard for an AnomalyDetected event

Shorter Learning Period for Tests

Use a short LearningPeriod in test environments so baselines are computed from a smaller window of data:

// In tests, use a short learning period
Anomaly: sentinel.AnomalyConfig{
    Enabled:        true,
    Sensitivity:    sentinel.AnomalySensitivityHigh, // Catch everything
    LearningPeriod: 1 * time.Hour,                  // Short window for tests
}

Unit Testing

You can test the anomaly detector programmatically by creating a store, seeding activity data, and calling CheckActivity directly:

anomaly_test.gogo
1func TestAnomalyDetection(t *testing.T) {
2    store := memory.New()
3    store.Migrate(context.Background())
4

5    pipe := pipeline.New(1000)
6    pipe.Start(1)
7    defer pipe.Stop()
8

9    geo := intelligence.NewGeoLocator(sentinel.GeoConfig{Enabled: false})
10

11    detector := intelligence.NewAnomalyDetector(store, pipe, geo, sentinel.AnomalyConfig{
12        Enabled:        true,
13        LearningPeriod: 7 * 24 * time.Hour,
14        Sensitivity:    sentinel.AnomalySensitivityMedium,
15        Checks: []sentinel.AnomalyCheckType{
16            sentinel.CheckOffHoursAccess,
17            sentinel.CheckUnusualAccess,
18            sentinel.CheckImpossibleTravel,
19        },
20    })
21

22    // Seed baseline: weekday 9-5 activity from US IP
23    for day := 1; day <= 7; day++ {
24        for hour := 9; hour <= 17; hour++ {
25            ts := time.Now().Add(-time.Duration(day) * 24 * time.Hour)
26            store.SaveUserActivity(ctx, &sentinel.UserActivity{
27                ID:        fmt.Sprintf("act-%d-%d", day, hour),
28                Timestamp: time.Date(ts.Year(), ts.Month(), ts.Day(), hour, 0, 0, 0, ts.Location()),
29                UserID:    "user1",
30                Path:      "/api/data",
31                Method:    "GET",
32                IP:        "10.0.0.1",
33                Country:   "US",
34            })
35        }
36    }
37

38    // Trigger anomaly: 3am access from a new country
39    err := detector.CheckActivity(ctx, &sentinel.UserActivity{
40        UserID:    "user1",
41        Timestamp: time.Date(2025, 1, 15, 3, 0, 0, 0, time.UTC),
42        Path:      "/admin/settings",
43        Method:    "GET",
44        IP:        "203.0.113.1",
45        Country:   "RU",
46    })
47    if err != nil {
48        t.Fatal(err)
49    }
50

51    // Verify a ThreatEvent was emitted via the pipeline
52    time.Sleep(100 * time.Millisecond)
53    // Check pipeline or store for AnomalyDetected event
54}

Timing in Tests

The anomaly detector emits threat events asynchronously through the pipeline. In unit tests, add a short time.Sleep (e.g., 100ms) after calling CheckActivity to allow the pipeline to process the event before asserting on results.

Next Steps

  • Configuration Reference — Full AnomalyConfig field reference
  • Auth Shield — Brute-force and credential stuffing protection
  • WAF — Signature-based attack detection to complement anomaly detection
  • Alerting — Get notified when anomalies are detected
  • Dashboard — View anomaly events and user baselines
Built with by JB