Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: SDK migration single resource scraping #2470

Merged
Merged
Show file tree
Hide file tree
Changes from 66 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
c1ee71a
implement logical flow for old and legacy Azure Monitor clients
hkfgo Apr 18, 2024
6a0d4f8
implement Azure Monitor integration under new SDK
hkfgo Apr 20, 2024
8e7ad70
properly implement handlers to track usage and ARM rate limiting metrics
hkfgo Apr 20, 2024
b16c000
use SDK-agnostic metric aggregation model
hkfgo Apr 24, 2024
92409d7
More SDK-agnostic refactoring to make Promitor build
hkfgo Apr 25, 2024
3943e03
Merge branch 'master' into feat/sdk-migration-single-resource-scraping
tomkerkhove Apr 26, 2024
ba74c67
log if using new SDK
hkfgo Apr 26, 2024
e3991be
Merge branch 'master' into feat/sdk-migration-single-resource-scraping
hkfgo Apr 26, 2024
3afc05f
more logging
hkfgo Apr 30, 2024
4a3b313
more logging
hkfgo Apr 30, 2024
91cf38e
use new SDK logging context for logger
hkfgo May 1, 2024
f40ab66
Update src/Promitor.Core.Scraping/Configuration/Model/MetricsDeclarat…
hkfgo May 1, 2024
a29807e
diff to trigger build again :(
hkfgo May 1, 2024
b676d83
switch to authentication factory for metric client authentication
hkfgo May 1, 2024
8e94aa6
github action image build
hkfgo May 2, 2024
76c2abf
Merge branch 'master' into feat/sdk-migration-single-resource-scraping
hkfgo May 2, 2024
d40c1d2
github action image build
hkfgo May 2, 2024
5dabb2e
Update templates-build-push-image.yml
hkfgo May 2, 2024
977beaa
handle no dimension configured case
hkfgo May 2, 2024
7a3239e
handle no dimension configured case
hkfgo May 2, 2024
727bed6
fix time range bug
hkfgo May 2, 2024
8da0f70
fix time range bug
hkfgo May 2, 2024
1d75e05
log filter query for debugging
hkfgo May 2, 2024
37dc13a
log filter query for debugging
hkfgo May 2, 2024
7bee0fd
use thread safe dictionary
hkfgo May 2, 2024
d2d3572
use thread safe dictionary
hkfgo May 3, 2024
3aef9a2
try ignore case for dimension
hkfgo May 3, 2024
724ac23
try ignore case for dimension
hkfgo May 3, 2024
f8c1957
try ignore case for dimension
hkfgo May 3, 2024
3d83a0e
try ignore case for dimension
hkfgo May 3, 2024
ebe6a8a
try ignore case for dimension
hkfgo May 3, 2024
819fe8c
try ignore case for dimension
hkfgo May 4, 2024
8d48752
try ignore case for dimension
hkfgo May 4, 2024
83c4d2d
try ignore case for dimension
hkfgo May 4, 2024
42639b4
try ignore case for dimension
hkfgo May 4, 2024
26ac926
try ignore case for dimension
hkfgo May 4, 2024
295fd9e
try ignore case for dimension
hkfgo May 4, 2024
6ed5825
try subtract time range
hkfgo May 4, 2024
b46ed3a
Throw exception on synchronous HTTP path
hkfgo May 8, 2024
02fde8a
address some small comments
hkfgo May 8, 2024
6b93808
address some small comments
hkfgo May 8, 2024
897e01d
address code factor issues
hkfgo May 8, 2024
a9e0226
remove unused imports
hkfgo May 8, 2024
958c953
use custom metric filter
hkfgo May 8, 2024
9f7e2cd
add deserlization test for SDk flag
hkfgo May 8, 2024
a4a13da
fix R#
hkfgo May 8, 2024
028187c
add diagnostic logging
hkfgo May 8, 2024
a05d86d
add diagnostic logging
hkfgo May 8, 2024
d6c6a01
remove more legacy sdk imports
hkfgo May 8, 2024
892303c
remove more legacy sdk imports
hkfgo May 8, 2024
65953b1
remove more legacy sdk imports
hkfgo May 8, 2024
369d94e
remove more legacy sdk imports
hkfgo May 8, 2024
8e2cca4
de-dupe filter conditions
hkfgo May 8, 2024
d4e5f60
de-dupe filter conditions
hkfgo May 8, 2024
3ea546d
de-dupe filter conditions
hkfgo May 8, 2024
1d89da7
de-dupe filter conditions
hkfgo May 8, 2024
79f2be7
de-dupe filter conditions
hkfgo May 8, 2024
b1796e0
fix empty check bug
hkfgo May 9, 2024
f3ddd12
fix empty check bug
hkfgo May 9, 2024
8e74685
fix empty check bug
hkfgo May 9, 2024
b91ecc1
fix empty check bug
hkfgo May 9, 2024
0398f47
fix empty check bug
hkfgo May 9, 2024
7042c64
case-insensitive dimension retrieval
hkfgo May 9, 2024
81ef6dc
case-insensitive dimension retrieval
hkfgo May 9, 2024
09b95b8
case-insensitive dimension retrieval
hkfgo May 9, 2024
67de88e
more clean up
hkfgo May 13, 2024
655964a
address small comments
hkfgo May 17, 2024
fd9d66e
update change log
hkfgo May 17, 2024
ec19284
add unit test for MetricAggregationType
hkfgo May 25, 2024
e9fd780
remove unused exception
hkfgo May 25, 2024
cfd55bd
remove unused exception
hkfgo May 25, 2024
be67705
remove unused exception
hkfgo May 25, 2024
1b245df
fix code format
hkfgo May 25, 2024
1426a64
fix code format
hkfgo May 25, 2024
d169eb8
remove excessive logging
hkfgo May 25, 2024
a414ae2
remove excessive logging
hkfgo May 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions .github/workflows/templates-build-push-image.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
name: Build and Push (Linux)
hkfgo marked this conversation as resolved.
Show resolved Hide resolved
on:
workflow_call:
workflow_dispatch:
inputs:
image_name:
required: true
Expand All @@ -15,6 +16,8 @@ jobs:
linux:
name: Build & Push (Linux)
runs-on: ubuntu-latest
permissions:
packages: write
steps:
- name: Checkout Code
uses: actions/checkout@v4
Expand All @@ -40,7 +43,7 @@ jobs:
uses: docker/login-action@v3
with:
registry: ghcr.io
username: tomkerkhove
username: hkfgo
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push preview image
Expand All @@ -49,5 +52,5 @@ jobs:
build-args: VERSION="${{ env.artifact_full_version }}"
context: ./src/
file: ./src/${{ inputs.project_name }}/Dockerfile.linux
tags: ${{ env.image_commit_uri }},${{ env.image_latest_uri }}
tags: ghcr.io/hkfgo/${{ env.image_commit_uri }},ghcr.io/hkfgo/${{ env.image_latest_uri }}
push: true
31 changes: 23 additions & 8 deletions src/Promitor.Agents.Scraper/AzureMonitorClientFactory.cs
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
using System.Collections.Generic;
using System.Collections.Concurrent;
using Microsoft.Azure.Management.ResourceManager.Fluent;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using Promitor.Core.Extensions;
using Promitor.Core.Metrics.Interfaces;
using Promitor.Core.Metrics.Sinks;
using Promitor.Core.Serialization.Enum;
using Promitor.Integrations.Azure.Authentication;
using Promitor.Integrations.AzureMonitor;
using Promitor.Integrations.AzureMonitor.Configuration;
Expand All @@ -14,7 +16,7 @@ namespace Promitor.Agents.Scraper
{
public class AzureMonitorClientFactory
{
private readonly Dictionary<string, AzureMonitorClient> _azureMonitorClients = new();
private readonly ConcurrentDictionary<string, IAzureMonitorClient> _azureMonitorClients = new();

/// <summary>
/// Provides an Azure Monitor client
Expand All @@ -29,24 +31,37 @@ public class AzureMonitorClientFactory
/// <param name="azureMonitorIntegrationConfiguration">Options for Azure Monitor integration</param>
/// <param name="azureMonitorLoggingConfiguration">Options for Azure Monitor logging</param>
/// <param name="loggerFactory">Factory to create loggers with</param>
public AzureMonitorClient CreateIfNotExists(AzureEnvironment cloud, string tenantId, string subscriptionId, MetricSinkWriter metricSinkWriter, IAzureScrapingSystemMetricsPublisher azureScrapingSystemMetricsPublisher, IMemoryCache resourceMetricDefinitionMemoryCache, IConfiguration configuration, IOptions<AzureMonitorIntegrationConfiguration> azureMonitorIntegrationConfiguration, IOptions<AzureMonitorLoggingConfiguration> azureMonitorLoggingConfiguration, ILoggerFactory loggerFactory)
{
/// <param name="useAzureMonitorSdk">Whether to use the new Azure.Monitor.Query package for queries</param>
public IAzureMonitorClient CreateIfNotExists(AzureCloud cloud, string tenantId, string subscriptionId, MetricSinkWriter metricSinkWriter, IAzureScrapingSystemMetricsPublisher azureScrapingSystemMetricsPublisher, IMemoryCache resourceMetricDefinitionMemoryCache, IConfiguration configuration, IOptions<AzureMonitorIntegrationConfiguration> azureMonitorIntegrationConfiguration, IOptions<AzureMonitorLoggingConfiguration> azureMonitorLoggingConfiguration, ILoggerFactory loggerFactory, bool useAzureMonitorSdk)
{
if (_azureMonitorClients.TryGetValue(subscriptionId, out var value))
{
return value;
}

var azureMonitorClient = CreateNewAzureMonitorClient(cloud, tenantId, subscriptionId, metricSinkWriter, azureScrapingSystemMetricsPublisher, resourceMetricDefinitionMemoryCache, configuration, azureMonitorIntegrationConfiguration, azureMonitorLoggingConfiguration, loggerFactory);
IAzureMonitorClient azureMonitorClient;
if (useAzureMonitorSdk) {
azureMonitorClient = CreateNewAzureMonitorQueryClient(cloud, tenantId, subscriptionId, metricSinkWriter, azureScrapingSystemMetricsPublisher, resourceMetricDefinitionMemoryCache, configuration, azureMonitorIntegrationConfiguration, azureMonitorLoggingConfiguration, loggerFactory);
} else {
azureMonitorClient = CreateNewLegacyAzureMonitorClient(cloud.GetAzureEnvironment(), tenantId, subscriptionId, metricSinkWriter, azureScrapingSystemMetricsPublisher, resourceMetricDefinitionMemoryCache, configuration, azureMonitorIntegrationConfiguration, azureMonitorLoggingConfiguration, loggerFactory);
}
_azureMonitorClients.TryAdd(subscriptionId, azureMonitorClient);

return azureMonitorClient;
}

private static AzureMonitorClient CreateNewAzureMonitorClient(AzureEnvironment cloud, string tenantId, string subscriptionId, MetricSinkWriter metricSinkWriter, IAzureScrapingSystemMetricsPublisher azureScrapingSystemMetricsPublisher, IMemoryCache resourceMetricDefinitionMemoryCache, IConfiguration configuration, IOptions<AzureMonitorIntegrationConfiguration> azureMonitorIntegrationConfiguration, IOptions<AzureMonitorLoggingConfiguration> azureMonitorLoggingConfiguration, ILoggerFactory loggerFactory)
private static LegacyAzureMonitorClient CreateNewLegacyAzureMonitorClient(AzureEnvironment cloud, string tenantId, string subscriptionId, MetricSinkWriter metricSinkWriter, IAzureScrapingSystemMetricsPublisher azureScrapingSystemMetricsPublisher, IMemoryCache resourceMetricDefinitionMemoryCache, IConfiguration configuration, IOptions<AzureMonitorIntegrationConfiguration> azureMonitorIntegrationConfiguration, IOptions<AzureMonitorLoggingConfiguration> azureMonitorLoggingConfiguration, ILoggerFactory loggerFactory)
{
var azureCredentials = AzureAuthenticationFactory.GetConfiguredAzureAuthentication(configuration);
var azureMonitorClient = new AzureMonitorClient(cloud, tenantId, subscriptionId, azureCredentials, metricSinkWriter, azureScrapingSystemMetricsPublisher, resourceMetricDefinitionMemoryCache, loggerFactory, azureMonitorIntegrationConfiguration, azureMonitorLoggingConfiguration);
return azureMonitorClient;
var azureMonitorClientLegacy = new LegacyAzureMonitorClient(cloud, tenantId, subscriptionId, azureCredentials, metricSinkWriter, azureScrapingSystemMetricsPublisher, resourceMetricDefinitionMemoryCache, loggerFactory, azureMonitorIntegrationConfiguration, azureMonitorLoggingConfiguration);
return azureMonitorClientLegacy;
}

private static AzureMonitorQueryClient CreateNewAzureMonitorQueryClient(AzureCloud cloud, string tenantId, string subscriptionId, MetricSinkWriter metricSinkWriter, IAzureScrapingSystemMetricsPublisher azureScrapingSystemMetricsPublisher, IMemoryCache resourceMetricDefinitionMemoryCache, IConfiguration configuration, IOptions<AzureMonitorIntegrationConfiguration> azureMonitorIntegrationConfiguration, IOptions<AzureMonitorLoggingConfiguration> azureMonitorLoggingConfiguration, ILoggerFactory loggerFactory)
{
var azureCredentials = AzureAuthenticationFactory.GetConfiguredAzureAuthentication(configuration);
var azureMonitorQueryClient = new AzureMonitorQueryClient(cloud, tenantId, subscriptionId, azureCredentials, metricSinkWriter, azureScrapingSystemMetricsPublisher, resourceMetricDefinitionMemoryCache, loggerFactory, azureMonitorIntegrationConfiguration, azureMonitorLoggingConfiguration);
return azureMonitorQueryClient;
}
}
}
3 changes: 2 additions & 1 deletion src/Promitor.Agents.Scraper/Docs/Open-Api.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 6 additions & 4 deletions src/Promitor.Agents.Scraper/Scheduling/ResourcesScrapingJob.cs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
using Microsoft.Extensions.Options;
using Promitor.Agents.Scraper.Discovery.Interfaces;
using Promitor.Core.Contracts;
using Promitor.Core.Extensions;
using Promitor.Core.Metrics.Interfaces;
using Promitor.Core.Metrics.Sinks;
using Promitor.Core.Scraping.Configuration.Model;
Expand Down Expand Up @@ -275,13 +276,14 @@ private async Task ScrapeMetric(ScrapeDefinition<IAzureResourceDefinition> scrap
var resourceSubscriptionId = !string.IsNullOrWhiteSpace(scrapeDefinition.Resource.SubscriptionId)
? scrapeDefinition.Resource.SubscriptionId
: _metricsDeclaration.AzureMetadata.SubscriptionId;
Logger.LogInformation("Parsed SDK Config {UseAzureMonitorSdk}", _metricsDeclaration.UseAzureMonitorSdk);
var azureMonitorClient = _azureMonitorClientFactory.CreateIfNotExists(_metricsDeclaration.AzureMetadata.Cloud, _metricsDeclaration.AzureMetadata.TenantId,
resourceSubscriptionId, _metricSinkWriter, _azureScrapingSystemMetricsPublisher, _resourceMetricDefinitionMemoryCache, _configuration,
_azureMonitorIntegrationConfiguration, _azureMonitorLoggingConfiguration, _loggerFactory);
_azureMonitorIntegrationConfiguration, _azureMonitorLoggingConfiguration, _loggerFactory, _metricsDeclaration.UseAzureMonitorSdk);

var tokenCredential = AzureAuthenticationFactory.GetTokenCredential(_metricsDeclaration.AzureMetadata.Cloud.ManagementEndpoint, _metricsDeclaration.AzureMetadata.TenantId,
AzureAuthenticationFactory.GetConfiguredAzureAuthentication(_configuration), new Uri(_metricsDeclaration.AzureMetadata.Cloud.AuthenticationEndpoint));
var logAnalyticsClient = new LogAnalyticsClient(_loggerFactory, _metricsDeclaration.AzureMetadata.Cloud, tokenCredential);
var tokenCredential = AzureAuthenticationFactory.GetTokenCredential(_metricsDeclaration.AzureMetadata.Cloud.GetAzureEnvironment().ManagementEndpoint, _metricsDeclaration.AzureMetadata.TenantId,
AzureAuthenticationFactory.GetConfiguredAzureAuthentication(_configuration), new Uri(_metricsDeclaration.AzureMetadata.Cloud.GetAzureEnvironment().AuthenticationEndpoint));
hkfgo marked this conversation as resolved.
Show resolved Hide resolved
var logAnalyticsClient = new LogAnalyticsClient(_loggerFactory, _metricsDeclaration.AzureMetadata.Cloud.GetAzureEnvironment(), tokenCredential);

var scraper = _metricScraperFactory.CreateScraper(scrapeDefinition.Resource.ResourceType, _metricSinkWriter, _azureScrapingSystemMetricsPublisher, azureMonitorClient, logAnalyticsClient);
await scraper.ScrapeAsync(scrapeDefinition);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ private void PlotAzureMetadataInAsciiTable(AzureMetadata metadata)
{
var asciiTable = CreateAzureMetadataAsciiTable();

asciiTable.AddRow(metadata.TenantId, metadata.SubscriptionId, metadata.ResourceGroupName, metadata.Cloud.Name.Humanize(LetterCasing.Title));
asciiTable.AddRow(metadata.TenantId, metadata.SubscriptionId, metadata.ResourceGroupName, metadata.Cloud.Humanize(LetterCasing.Title));

AnsiConsole.Write(asciiTable);
}
Expand Down
3 changes: 1 addition & 2 deletions src/Promitor.Core.Scraping/AzureMonitorScraper.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
using System.Linq;
using System.Threading.Tasks;
using GuardNet;
using Microsoft.Azure.Management.Monitor.Fluent.Models;
using Microsoft.Extensions.Logging;
using Promitor.Core.Contracts;
using Promitor.Core.Metrics;
Expand All @@ -29,7 +28,7 @@ protected AzureMonitorScraper(ScraperConfiguration scraperConfiguration) :
}

/// <inheritdoc />
protected override async Task<ScrapeResult> ScrapeResourceAsync(string subscriptionId, ScrapeDefinition<IAzureResourceDefinition> scrapeDefinition, TResourceDefinition resourceDefinition, AggregationType aggregationType, TimeSpan aggregationInterval)
protected override async Task<ScrapeResult> ScrapeResourceAsync(string subscriptionId, ScrapeDefinition<IAzureResourceDefinition> scrapeDefinition, TResourceDefinition resourceDefinition, PromitorMetricAggregationType aggregationType, TimeSpan aggregationInterval)
{
Guard.NotNull(scrapeDefinition, nameof(scrapeDefinition));
Guard.NotNull(scrapeDefinition.AzureMetricConfiguration, nameof(scrapeDefinition.AzureMetricConfiguration));
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
using Microsoft.Azure.Management.ResourceManager.Fluent;
using Promitor.Core.Serialization.Enum;

namespace Promitor.Core.Scraping.Configuration.Model
{
Expand All @@ -7,6 +7,6 @@ public class AzureMetadata
public string ResourceGroupName { get; set; }
public string SubscriptionId { get; set; }
public string TenantId { get; set; }
public AzureEnvironment Cloud { get; set; }
public AzureCloud Cloud { get; set; }
}
}
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using Microsoft.Azure.Management.Monitor.Fluent.Models;

using Promitor.Core.Metrics;

namespace Promitor.Core.Scraping.Configuration.Model
{
Expand All @@ -7,6 +8,6 @@ public class MetricAggregation : Aggregation
/// <summary>
/// Type of aggregation to query the Azure Monitor metric
/// </summary>
public AggregationType Type { get; set; }
public PromitorMetricAggregationType Type { get; set; }
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ namespace Promitor.Core.Scraping.Configuration.Model
{
public class MetricsDeclaration
{
public bool UseAzureMonitorSdk { get; set; } = true;
public AzureMetadata AzureMetadata { get; set; }
public MetricDefaults MetricDefaults { get; set; } = new MetricDefaults();
public List<MetricDefinition> Metrics { get; set; } = new List<MetricDefinition>();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
using System;
using System.Collections.Generic;
using Microsoft.Azure.Management.ResourceManager.Fluent;
using Microsoft.Extensions.Logging;
using Promitor.Core.Extensions;
using Promitor.Core.Scraping.Configuration.Serialization.v1.Model;
Expand All @@ -20,18 +19,19 @@ public AzureMetadataDeserializer(ILogger<AzureMetadataDeserializer> logger) : ba
Map(metadata => metadata.ResourceGroupName)
.IsRequired();
Map(metadata => metadata.Cloud)
.WithDefault(AzureEnvironment.AzureGlobalCloud)
.WithDefault(AzureCloud.Global)
.MapUsing(DetermineAzureCloud);
}

// TODO: validate cloud configuration in a SDK-agnostic way
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you open an issue and link to it here please?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used azureCloud.DetermineMetricsClientAudience() for validation, see comment above

private object DetermineAzureCloud(string rawAzureCloud, KeyValuePair<YamlNode, YamlNode> nodePair, IErrorReporter errorReporter)
{
if (Enum.TryParse<AzureCloud>(rawAzureCloud, out var azureCloud))
{
try
{
var azureEnvironment = azureCloud.GetAzureEnvironment();
return azureEnvironment;
azureCloud.DetermineMetricsClientAudience();
return azureCloud;
tomkerkhove marked this conversation as resolved.
Show resolved Hide resolved
}
catch (ArgumentOutOfRangeException)
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ public V1Deserializer(IDeserializer<AzureMetadataV1> azureMetadataDeserializer,
Map(definition => definition.Version)
.IsRequired()
.MapUsing(GetVersion);
Map(definition => definition.UseAzureMonitorSdk)
.WithDefault(true);
Map(definition => definition.AzureMetadata)
.IsRequired()
.MapUsingDeserializer(azureMetadataDeserializer);
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
using Microsoft.Azure.Management.ResourceManager.Fluent;
using Promitor.Core.Serialization.Enum;

namespace Promitor.Core.Scraping.Configuration.Serialization.v1.Model
{
Expand All @@ -7,6 +7,6 @@ public class AzureMetadataV1
public string TenantId { get; set; }
public string SubscriptionId { get; set; }
public string ResourceGroupName { get; set; }
public AzureEnvironment Cloud { get; set; }
public AzureCloud Cloud { get; set; }
}
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
using System;
using Microsoft.Azure.Management.Monitor.Fluent.Models;
using Promitor.Core.Metrics;

namespace Promitor.Core.Scraping.Configuration.Serialization.v1.Model
{
Expand All @@ -16,6 +16,6 @@ public class MetricAggregationV1
/// <summary>
/// Type of aggregation to query the Azure Monitor metric.
/// </summary>
public AggregationType? Type { get; set; }
public PromitorMetricAggregationType? Type { get; set; }
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ namespace Promitor.Core.Scraping.Configuration.Serialization.v1.Model
public class MetricsDeclarationV1
{
public string Version { get; set; } = SpecVersion.v1.ToString();
public bool UseAzureMonitorSdk { get; set; } = true;
public AzureMetadataV1 AzureMetadata { get; set; }
public MetricDefaultsV1 MetricDefaults { get; set; }
public IReadOnlyCollection<MetricDefinitionV1> Metrics { get; set; }
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,119 +31,119 @@
/// <param name="azureScrapingSystemMetricsPublisher">Collector to send metrics related to the runtime</param>
/// <param name="azureMonitorClient">Client to interact with Azure Monitor</param>
/// <param name="logAnalyticsClient">Client to interact with Log Analytics</param>
public IScraper<IAzureResourceDefinition> CreateScraper(ResourceType metricDefinitionResourceType, MetricSinkWriter metricSinkWriter, IAzureScrapingSystemMetricsPublisher azureScrapingSystemMetricsPublisher, AzureMonitorClient azureMonitorClient, LogAnalyticsClient logAnalyticsClient)
public IScraper<IAzureResourceDefinition> CreateScraper(ResourceType metricDefinitionResourceType, MetricSinkWriter metricSinkWriter, IAzureScrapingSystemMetricsPublisher azureScrapingSystemMetricsPublisher, IAzureMonitorClient azureMonitorClient, LogAnalyticsClient logAnalyticsClient)
{
var scraperConfiguration = new ScraperConfiguration(azureMonitorClient, logAnalyticsClient, metricSinkWriter, azureScrapingSystemMetricsPublisher, _logger);

switch (metricDefinitionResourceType)
{
case ResourceType.ApiManagement:
return new ApiManagementScraper(scraperConfiguration);
case ResourceType.ApplicationGateway:
return new ApplicationGatewayScraper(scraperConfiguration);
case ResourceType.ApplicationInsights:
return new ApplicationInsightsScraper(scraperConfiguration);
case ResourceType.AppPlan:
return new AppPlanScraper(scraperConfiguration);
case ResourceType.AutomationAccount:
return new AutomationAccountScraper(scraperConfiguration);
case ResourceType.BlobStorage:
return new BlobStorageScraper(scraperConfiguration);
case ResourceType.Cdn:
return new CdnScraper(scraperConfiguration);
case ResourceType.ContainerInstance:
return new ContainerInstanceScraper(scraperConfiguration);
case ResourceType.ContainerRegistry:
return new ContainerRegistryScraper(scraperConfiguration);
case ResourceType.CosmosDb:
return new CosmosDbScraper(scraperConfiguration);
case ResourceType.DataExplorerCluster:
return new DataExplorerClusterScraper(scraperConfiguration);
case ResourceType.DataFactory:
return new DataFactoryScraper(scraperConfiguration);
case ResourceType.DataShare:
return new DataShareScraper(scraperConfiguration);
case ResourceType.DeviceProvisioningService:
return new DeviceProvisioningServiceScraper(scraperConfiguration);
case ResourceType.EventHubs:
return new EventHubsScraper(scraperConfiguration);
case ResourceType.ExpressRouteCircuit:
return new ExpressRouteCircuitScraper(scraperConfiguration);
case ResourceType.FileStorage:
return new FileStorageScraper(scraperConfiguration);
case ResourceType.FrontDoor:
return new FrontDoorScraper(scraperConfiguration);
case ResourceType.FunctionApp:
return new FunctionAppScraper(scraperConfiguration);
case ResourceType.Generic:
return new GenericScraper(scraperConfiguration);
case ResourceType.IoTHub:
return new IoTHubScraper(scraperConfiguration);
case ResourceType.KeyVault:
return new KeyVaultScraper(scraperConfiguration);
case ResourceType.KubernetesService:
return new KubernetesServiceScraper(scraperConfiguration);
case ResourceType.LoadBalancer:
return new LoadBalancerScraper(scraperConfiguration);
case ResourceType.LogAnalytics:
return new LogAnalyticsScraper(scraperConfiguration);
case ResourceType.LogicApp:
return new LogicAppScraper(scraperConfiguration);
case ResourceType.MariaDb:
return new MariaDbScraper(scraperConfiguration);
case ResourceType.MonitorAutoscale:
return new MonitorAutoscaleScraper(scraperConfiguration);
case ResourceType.MySql:
return new MySqlScraper(scraperConfiguration);
case ResourceType.NatGateway:
return new NatGatewayScraper(scraperConfiguration);
case ResourceType.NetworkGateway:
return new NetworkGatewayScraper(scraperConfiguration);
case ResourceType.NetworkInterface:
return new NetworkInterfaceScraper(scraperConfiguration);
case ResourceType.PostgreSql:
return new PostgreSqlScraper(scraperConfiguration);
case ResourceType.PowerBiDedicated:
return new PowerBiDedicatedScraper(scraperConfiguration);
case ResourceType.PublicIpAddress:
return new PublicIpAddressScraper(scraperConfiguration);
case ResourceType.RedisCache:
return new RedisCacheScraper(scraperConfiguration);
case ResourceType.RedisEnterpriseCache:
return new RedisEnterpriseCacheScraper(scraperConfiguration);
case ResourceType.ServiceBusNamespace:
return new ServiceBusNamespaceScraper(scraperConfiguration);
case ResourceType.SqlDatabase:
return new SqlDatabaseScraper(scraperConfiguration);
case ResourceType.SqlElasticPool:
return new SqlElasticPoolScraper(scraperConfiguration);
case ResourceType.SqlManagedInstance:
return new SqlManagedInstanceScraper(scraperConfiguration);
case ResourceType.SqlServer:
return new SqlServerScraper(scraperConfiguration);
case ResourceType.StorageAccount:
return new StorageAccountScraper(scraperConfiguration);
case ResourceType.StorageQueue:
return new StorageQueueScraper(scraperConfiguration);
case ResourceType.SynapseApacheSparkPool:
return new SynapseApacheSparkPoolScraper(scraperConfiguration);
case ResourceType.SynapseSqlPool:
return new SynapseSqlPoolScraper(scraperConfiguration);
case ResourceType.SynapseWorkspace:
return new SynapseWorkspaceScraper(scraperConfiguration);
case ResourceType.TrafficManager:
return new TrafficManagerScraper(scraperConfiguration);
case ResourceType.VirtualMachine:
return new VirtualMachineScraper(scraperConfiguration);
case ResourceType.VirtualMachineScaleSet:
return new VirtualMachineScaleSetScraper(scraperConfiguration);
case ResourceType.VirtualNetwork:
return new VirtualNetworkScraper(scraperConfiguration);
case ResourceType.WebApp:
return new WebAppScraper(scraperConfiguration);
default:
throw new ArgumentOutOfRangeException(nameof(metricDefinitionResourceType), metricDefinitionResourceType, "Matching scraper not found");
}
}
}

Check notice on line 148 in src/Promitor.Core.Scraping/Factories/MetricScraperFactory.cs

View check run for this annotation

codefactor.io / CodeFactor

src/Promitor.Core.Scraping/Factories/MetricScraperFactory.cs#L34-L148

Complex Method
}
Loading
Loading