#360 Routing based on request header (#1312)

* routing based on headers (all specified headers must match)

* routing based on headers for aggregated routes

* unit tests and small modifications

* find placeholders in header templates

* match upstream headers to header templates

* find placeholders name and values, fix regex for finding placeholders values

* fix unit tests

* change header placeholder pattern

* unit tests

* unit tests

* unit tests

* unit tests

* extend validation with checking upstreamheadertemplates, acceptance tests for cases from the issue

* update docs and minor changes

* SA1649 File name should match first type name

* Fix compilation errors by code review after resolving conflicts

* Fix warnings

* File-scoped namespaces

* File-scoped namespace

* Target-typed 'new' expressions (C# 9).
https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-9.0/target-typed-new

* IDE1006 Naming rule violation: These words must begin with upper case characters: should_*

* Target-typed 'new' expressions (C# 9).
https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-9.0/target-typed-new

* Fix build errors

* DownstreamRouteBuilder

* AggregatesCreator

* IUpstreamHeaderTemplatePatternCreator, RoutesCreator

* UpstreamHeaderTemplatePatternCreator

* FileAggregateRoute

* FileAggregateRoute

* FileRoute

* Route, IRoute

* FileConfigurationFluentValidator

* OcelotBuilder

* DownstreamRouteCreator

* DownstreamRouteFinder

* HeaderMatcher

* DownstreamRouteFinderMiddleware

* UpstreamHeaderTemplate

* Routing folder

* RoutingBasedOnHeadersTests

* Refactor acceptance tests

* AAA pattern in unit tests

* CS8936: Feature 'collection expressions' is not available in C# 10.0.
Please use language version 12.0 or greater.

* Code review by @RaynaldM

* Convert facts to one `Theory`

* AAA pattern

* Add traits

* Update routing.rst

Check grammar and style

* Update docs

---------

Co-authored-by: raman-m <[email protected]>

Author: jlukawska

docs/features/configuration.rst

@@ -19,9 +19,11 @@ Here is an example Route configuration. You don't need to set all of these thing
 .. code-block:: json
 
   {
-    "DownstreamPathTemplate": "/",
     "UpstreamPathTemplate": "/",
+    "UpstreamHeaderTemplates": {}, // dictionary
+    "UpstreamHost": "",
     "UpstreamHttpMethod": [ "Get" ],
+    "DownstreamPathTemplate": "/",
     "DownstreamHttpMethod": "",
     "DownstreamHttpVersion": "",
     "AddHeadersToRequest": {},
@@ -37,7 +39,7 @@ Here is an example Route configuration. You don't need to set all of these thing
     "ServiceName": "",
     "DownstreamScheme": "http",
     "DownstreamHostAndPorts": [
-      { "Host": "localhost", "Port": 51876 }
+      { "Host": "localhost", "Port": 12345 }
     ],
     "QoSOptions": {
       "ExceptionsAllowedBeforeBreaking": 0,
@@ -70,7 +72,8 @@ Here is an example Route configuration. You don't need to set all of these thing
     }
   }
 
-More information on how to use these options is below.
+The actual Route schema for properties can be found in the C# `FileRoute <https://github.com/ThreeMammals/Ocelot/blob/main/src/Ocelot/Configuration/File/FileRoute.cs>`_ class.
+If you're interested in learning more about how to utilize these options, read below!
 
 Multiple Environments
 ---------------------

docs/features/routing.rst

@@ -154,6 +154,58 @@ The Route above will only be matched when the ``Host`` header value is ``somedom
 If you do not set **UpstreamHost** on a Route then any ``Host`` header will match it.
 This means that if you have two Routes that are the same, apart from the **UpstreamHost**, where one is null and the other set Ocelot will favour the one that has been set. 
 
+.. _routing-upstream-headers:
+
+Upstream Headers [#f3]_
+-----------------------
+
+In addition to routing by ``UpstreamPathTemplate``, you can also define ``UpstreamHeaderTemplates``.
+For a route to match, all headers specified in this dictionary object must be present in the request headers.
+
+.. code-block:: json
+
+  {
+    // ...
+    "UpstreamPathTemplate": "/",
+    "UpstreamHttpMethod": [ "Get" ],
+    "UpstreamHeaderTemplates": { // dictionary
+      "country": "uk", // 1st header
+      "version": "v1"  // 2nd header
+    }
+  }
+
+In this scenario, the route will only match if a request includes both headers with the specified values.
+
+Header placeholders
+^^^^^^^^^^^^^^^^^^^
+
+Let's explore a more intriguing scenario where placeholders can be effectively utilized within your ``UpstreamHeaderTemplates``.
+
+Consider the following approach using the special placeholder format ``{header:placeholdername}``:
+
+.. code-block:: json
+
+  {
+    "DownstreamPathTemplate": "/{versionnumber}/api", // with placeholder
+    "DownstreamScheme": "https",
+    "DownstreamHostAndPorts": [
+      { "Host": "10.0.10.1", "Port": 80 }
+    ],
+    "UpstreamPathTemplate": "/api",
+    "UpstreamHttpMethod": [ "Get" ],
+    "UpstreamHeaderTemplates": {
+      "version": "{header:versionnumber}" // 'header:' prefix vs placeholder
+    }
+  }
+
+In this scenario, the entire value of the request header "**version**" is inserted into the ``DownstreamPathTemplate``.
+If necessary, a more intricate upstream header template can be specified, using placeholders such as ``version-{header:version}_country-{header:country}``.
+
+  **Note 1**: Placeholders are not required in ``DownstreamPathTemplate``.
+  This scenario can be utilized to mandate a specific header regardless of its value.
+
+  **Note 2**: Additionally, the ``UpstreamHeaderTemplates`` dictionary options are applicable for :doc:`../features/requestaggregation` as well.
+
 Priority
 --------
 
@@ -294,7 +346,7 @@ Here are two user scenarios.
 
 .. _routing-security-options:
 
-Security Options [#f3]_
+Security Options [#f4]_
 -----------------------
 
 Ocelot allows you to manage multiple patterns for allowed/blocked IPs using the `IPAddressRange <https://github.com/jsakamoto/ipaddressrange>`_ package
@@ -326,7 +378,7 @@ The current patterns managed are the following:
 
 .. _routing-dynamic:
 
-Dynamic Routing [#f4]_
+Dynamic Routing [#f5]_
 ----------------------
 
 The idea is to enable dynamic routing when using a :doc:`../features/servicediscovery` provider so you don't have to provide the Route config.
@@ -336,5 +388,6 @@ See the :ref:`sd-dynamic-routing` docs if this sounds interesting to you.
 
 .. [#f1] ":ref:`routing-empty-placeholders`" feature is available starting in version `23.0 <https://github.com/ThreeMammals/Ocelot/releases/tag/23.0.0>`_, see issue `748 <https://github.com/ThreeMammals/Ocelot/issues/748>`_ and the `23.0 <https://github.com/ThreeMammals/Ocelot/releases/tag/23.0.0>`__ release notes for details.
 .. [#f2] ":ref:`routing-upstream-host`" feature was requested as part of `issue 216 <https://github.com/ThreeMammals/Ocelot/pull/216>`_.
-.. [#f3] ":ref:`routing-security-options`" feature was requested as part of `issue 628 <https://github.com/ThreeMammals/Ocelot/issues/628>`_ (of `12.0.1 <https://github.com/ThreeMammals/Ocelot/releases/tag/12.0.1>`_ version), then redesigned and improved by `issue 1400 <https://github.com/ThreeMammals/Ocelot/issues/1400>`_, and published in version `20.0 <https://github.com/ThreeMammals/Ocelot/releases/tag/20.0.0>`_ docs.
-.. [#f4] ":ref:`routing-dynamic`" feature was requested as part of `issue 340 <https://github.com/ThreeMammals/Ocelot/issues/340>`_. Complete reference: :ref:`sd-dynamic-routing`.
+.. [#f3] ":ref:`routing-upstream-headers`" feature was proposed in `issue 360 <https://github.com/ThreeMammals/Ocelot/issues/360>`_, and released in version `24.0 <https://github.com/ThreeMammals/Ocelot/releases/tag/24.0.0>`_.
+.. [#f4] ":ref:`routing-security-options`" feature was requested as part of `issue 628 <https://github.com/ThreeMammals/Ocelot/issues/628>`_ (of `12.0.1 <https://github.com/ThreeMammals/Ocelot/releases/tag/12.0.1>`_ version), then redesigned and improved by `issue 1400 <https://github.com/ThreeMammals/Ocelot/issues/1400>`_, and published in version `20.0 <https://github.com/ThreeMammals/Ocelot/releases/tag/20.0.0>`_ docs.
+.. [#f5] ":ref:`routing-dynamic`" feature was requested as part of `issue 340 <https://github.com/ThreeMammals/Ocelot/issues/340>`_. Complete reference: :ref:`sd-dynamic-routing`.

src/Ocelot/Configuration/Builder/DownstreamRouteBuilder.cs

@@ -40,13 +40,14 @@ public class DownstreamRouteBuilder
     private SecurityOptions _securityOptions;
     private string _downstreamHttpMethod;
     private Version _downstreamHttpVersion;
+    private Dictionary<string, UpstreamHeaderTemplate> _upstreamHeaders;
 
     public DownstreamRouteBuilder()
     {
-        _downstreamAddresses = new List<DownstreamHostAndPort>();
-        _delegatingHandlers = new List<string>();
-        _addHeadersToDownstream = new List<AddHeader>();
-        _addHeadersToUpstream = new List<AddHeader>();
+        _downstreamAddresses = new();
+        _delegatingHandlers = new();
+        _addHeadersToDownstream = new();
+        _addHeadersToUpstream = new();
     }
 
     public DownstreamRouteBuilder WithDownstreamAddresses(List<DownstreamHostAndPort> downstreamAddresses)
@@ -87,7 +88,9 @@ public DownstreamRouteBuilder WithUpstreamPathTemplate(UpstreamPathTemplate inpu
 
     public DownstreamRouteBuilder WithUpstreamHttpMethod(List<string> input)
     {
-        _upstreamHttpMethod = input.Count == 0 ? new List<HttpMethod>() : input.Select(x => new HttpMethod(x.Trim())).ToList();
+        _upstreamHttpMethod = input.Count > 0
+            ? input.Select(x => new HttpMethod(x.Trim())).ToList()
+            : new();
         return this;
     }
 
@@ -259,6 +262,12 @@ public DownstreamRouteBuilder WithDownstreamHttpVersion(Version downstreamHttpVe
         return this;
     }
 
+    public DownstreamRouteBuilder WithUpstreamHeaders(Dictionary<string, UpstreamHeaderTemplate> input)
+    {
+        _upstreamHeaders = input;
+        return this;
+    }
+
     public DownstreamRoute Build()
     {
         return new DownstreamRoute(
@@ -295,6 +304,7 @@ public DownstreamRoute Build()
             _dangerousAcceptAnyServerCertificateValidator,
             _securityOptions,
             _downstreamHttpMethod,
-            _downstreamHttpVersion);
+            _downstreamHttpVersion,
+            _upstreamHeaders);
     }
 }

src/Ocelot/Configuration/Builder/RouteBuilder.cs

@@ -10,7 +10,8 @@ public class RouteBuilder
         private string _upstreamHost;
         private List<DownstreamRoute> _downstreamRoutes;
         private List<AggregateRouteConfig> _downstreamRoutesConfig;
-        private string _aggregator;
+        private string _aggregator;
+        private IDictionary<string, UpstreamHeaderTemplate> _upstreamHeaders;
 
         public RouteBuilder()
         {
@@ -58,6 +59,12 @@ public RouteBuilder WithAggregator(string aggregator)
         {
             _aggregator = aggregator;
             return this;
+        }
+
+        public RouteBuilder WithUpstreamHeaders(IDictionary<string, UpstreamHeaderTemplate> upstreamHeaders)
+        {
+            _upstreamHeaders = upstreamHeaders;
+            return this;
         }
 
         public Route Build()
@@ -68,7 +75,8 @@ public Route Build()
                 _upstreamHttpMethod,
                 _upstreamTemplatePattern,
                 _upstreamHost,
-                _aggregator
+                _aggregator,
+                _upstreamHeaders
                 );
         }
     }

src/Ocelot/Configuration/Creator/AggregatesCreator.cs

@@ -5,11 +5,13 @@ namespace Ocelot.Configuration.Creator
 {
     public class AggregatesCreator : IAggregatesCreator
     {
-        private readonly IUpstreamTemplatePatternCreator _creator;
+        private readonly IUpstreamTemplatePatternCreator _creator;
+        private readonly IUpstreamHeaderTemplatePatternCreator _headerCreator;
 
-        public AggregatesCreator(IUpstreamTemplatePatternCreator creator)
+        public AggregatesCreator(IUpstreamTemplatePatternCreator creator, IUpstreamHeaderTemplatePatternCreator headerCreator)
         {
-            _creator = creator;
+            _creator = creator;
+            _headerCreator = headerCreator;
         }
 
         public List<Route> Create(FileConfiguration fileConfiguration, List<Route> routes)
@@ -35,15 +37,17 @@ private Route SetUpAggregateRoute(IEnumerable<Route> routes, FileAggregateRoute
                 applicableRoutes.Add(downstreamRoute);
             }
 
-            var upstreamTemplatePattern = _creator.Create(aggregateRoute);
+            var upstreamTemplatePattern = _creator.Create(aggregateRoute);
+            var upstreamHeaderTemplates = _headerCreator.Create(aggregateRoute);
 
             var route = new RouteBuilder()
                 .WithUpstreamHttpMethod(aggregateRoute.UpstreamHttpMethod)
                 .WithUpstreamPathTemplate(upstreamTemplatePattern)
                 .WithDownstreamRoutes(applicableRoutes)
                 .WithAggregateRouteConfig(aggregateRoute.RouteKeysConfig)
                 .WithUpstreamHost(aggregateRoute.UpstreamHost)
-                .WithAggregator(aggregateRoute.Aggregator)
+                .WithAggregator(aggregateRoute.Aggregator)
+                .WithUpstreamHeaders(upstreamHeaderTemplates)
                 .Build();
 
             return route;

src/Ocelot/Configuration/Creator/IUpstreamHeaderTemplatePatternCreator.cs

@@ -0,0 +1,17 @@
+using Ocelot.Configuration.File;
+using Ocelot.Values;
+
+namespace Ocelot.Configuration.Creator;
+
+/// <summary>
+/// Ocelot feature: <see href="https://github.com/ThreeMammals/Ocelot/blob/develop/docs/features/routing.rst#upstream-headers">Routing based on request header</see>.
+/// </summary>
+public interface IUpstreamHeaderTemplatePatternCreator
+{
+    /// <summary>
+    /// Creates upstream templates based on route headers.
+    /// </summary>
+    /// <param name="route">The route info.</param>
+    /// <returns>An <see cref="IDictionary{TKey, TValue}"/> object where TKey is <see langword="string"/>, TValue is <see cref="UpstreamHeaderTemplate"/>.</returns>
+    IDictionary<string, UpstreamHeaderTemplate> Create(IRoute route);
+}

src/Ocelot/Configuration/Creator/RoutesCreator.cs

@@ -1,4 +1,4 @@
-using Ocelot.Configuration.Builder;
+using Ocelot.Configuration.Builder;
 using Ocelot.Configuration.File;
 
 namespace Ocelot.Configuration.Creator
@@ -9,6 +9,7 @@ public class RoutesCreator : IRoutesCreator
         private readonly IClaimsToThingCreator _claimsToThingCreator;
         private readonly IAuthenticationOptionsCreator _authOptionsCreator;
         private readonly IUpstreamTemplatePatternCreator _upstreamTemplatePatternCreator;
+        private readonly IUpstreamHeaderTemplatePatternCreator _upstreamHeaderTemplatePatternCreator;
         private readonly IRequestIdKeyCreator _requestIdKeyCreator;
         private readonly IQoSOptionsCreator _qosOptionsCreator;
         private readonly IRouteOptionsCreator _fileRouteOptionsCreator;
@@ -36,7 +37,8 @@ public RoutesCreator(
             ILoadBalancerOptionsCreator loadBalancerOptionsCreator,
             IRouteKeyCreator routeKeyCreator,
             ISecurityOptionsCreator securityOptionsCreator,
-            IVersionCreator versionCreator)
+            IVersionCreator versionCreator,
+            IUpstreamHeaderTemplatePatternCreator upstreamHeaderTemplatePatternCreator)
         {
             _routeKeyCreator = routeKeyCreator;
             _loadBalancerOptionsCreator = loadBalancerOptionsCreator;
@@ -54,6 +56,7 @@ public RoutesCreator(
             _loadBalancerOptionsCreator = loadBalancerOptionsCreator;
             _securityOptionsCreator = securityOptionsCreator;
             _versionCreator = versionCreator;
+            _upstreamHeaderTemplatePatternCreator = upstreamHeaderTemplatePatternCreator;
         }
 
         public List<Route> Create(FileConfiguration fileConfiguration)
@@ -149,12 +152,14 @@ private DownstreamRoute SetUpDownstreamRoute(FileRoute fileRoute, FileGlobalConf
         private Route SetUpRoute(FileRoute fileRoute, DownstreamRoute downstreamRoutes)
         {
             var upstreamTemplatePattern = _upstreamTemplatePatternCreator.Create(fileRoute);
+            var upstreamHeaderTemplates = _upstreamHeaderTemplatePatternCreator.Create(fileRoute);
 
             var route = new RouteBuilder()
                 .WithUpstreamHttpMethod(fileRoute.UpstreamHttpMethod)
                 .WithUpstreamPathTemplate(upstreamTemplatePattern)
                 .WithDownstreamRoute(downstreamRoutes)
                 .WithUpstreamHost(fileRoute.UpstreamHost)
+                .WithUpstreamHeaders(upstreamHeaderTemplates)
                 .Build();
 
             return route;

src/Ocelot/Configuration/Creator/UpstreamHeaderTemplatePatternCreator.cs

@@ -0,0 +1,50 @@
+using Ocelot.Configuration.File;
+using Ocelot.Values;
+
+namespace Ocelot.Configuration.Creator;
+
+/// <summary>
+/// Default creator of upstream templates based on route headers.
+/// </summary>
+/// <remarks>Ocelot feature: Routing based on request header.</remarks>
+public partial class UpstreamHeaderTemplatePatternCreator : IUpstreamHeaderTemplatePatternCreator
+{
+    private const string PlaceHolderPattern = @"(\{header:.*?\})";
+#if NET7_0_OR_GREATER
+    [GeneratedRegex(PlaceHolderPattern, RegexOptions.IgnoreCase | RegexOptions.Singleline, "en-US")]
+    private static partial Regex RegExPlaceholders();
+#else
+    private static readonly Regex RegExPlaceholdersVar = new(PlaceHolderPattern, RegexOptions.IgnoreCase | RegexOptions.Singleline, TimeSpan.FromMilliseconds(1000));
+    private static Regex RegExPlaceholders() => RegExPlaceholdersVar;
+#endif
+
+    public IDictionary<string, UpstreamHeaderTemplate> Create(IRoute route)
+    {
+        var result = new Dictionary<string, UpstreamHeaderTemplate>();
+
+        foreach (var headerTemplate in route.UpstreamHeaderTemplates)
+        {
+            var headerTemplateValue = headerTemplate.Value;
+            var matches = RegExPlaceholders().Matches(headerTemplateValue);
+
+            if (matches.Count > 0)
+            {
+                var placeholders = matches.Select(m => m.Groups[1].Value).ToArray();
+                for (int i = 0; i < placeholders.Length; i++)
+                {
+                    var indexOfPlaceholder = headerTemplateValue.IndexOf(placeholders[i]);
+                    var placeholderName = placeholders[i][8..^1]; // remove "{header:" and "}"
+                    headerTemplateValue = headerTemplateValue.Replace(placeholders[i], $"(?<{placeholderName}>.+)");
+                }
+            }
+
+            var template = route.RouteIsCaseSensitive
+                ? $"^{headerTemplateValue}$"
+                : $"^(?i){headerTemplateValue}$"; // ignore case
+
+            result.Add(headerTemplate.Key, new(template, headerTemplate.Value));
+        }
+
+        return result;
+    }
+}

src/Ocelot/Configuration/DownstreamRoute.cs

@@ -39,7 +39,8 @@ public DownstreamRoute(
             bool dangerousAcceptAnyServerCertificateValidator,
             SecurityOptions securityOptions,
             string downstreamHttpMethod,
-            Version downstreamHttpVersion)
+            Version downstreamHttpVersion,
+            Dictionary<string, UpstreamHeaderTemplate> upstreamHeaders)
         {
             DangerousAcceptAnyServerCertificateValidator = dangerousAcceptAnyServerCertificateValidator;
             AddHeadersToDownstream = addHeadersToDownstream;
@@ -74,7 +75,8 @@ public DownstreamRoute(
             AddHeadersToUpstream = addHeadersToUpstream;
             SecurityOptions = securityOptions;
             DownstreamHttpMethod = downstreamHttpMethod;
-            DownstreamHttpVersion = downstreamHttpVersion;
+            DownstreamHttpVersion = downstreamHttpVersion;
+            UpstreamHeaders = upstreamHeaders ?? new();
         }
 
         public string Key { get; }
@@ -110,6 +112,7 @@ public DownstreamRoute(
         public bool DangerousAcceptAnyServerCertificateValidator { get; }
         public SecurityOptions SecurityOptions { get; }
         public string DownstreamHttpMethod { get; }
-        public Version DownstreamHttpVersion { get; }
+        public Version DownstreamHttpVersion { get; }
+        public Dictionary<string, UpstreamHeaderTemplate> UpstreamHeaders { get; }
     }
 }