5GMS Consumption reporting: Add support to Media Session Handler #38

shilinding · 2023-09-01T10:29:52Z

Get consumption report from Media Stream Handler and post to AF via M5.

Resolves #3.

dsilhavy · 2023-09-04T06:52:54Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

@@ -123,6 +139,15 @@ class MediaSessionHandlerMessengerService() : Service() {
                    val resource =
                        handleServiceAccessResponse(response, sendingUid, provisioningSessionId)

+                    // initialize Retrofit for ConsumptionReporting
+                    currentServiceAccessInformation = resource


We are maintaining multiple concurrent sessions in the Media Session Handler. This means that multiple applications can be connected to the same Media Session Handler (see handling in handleServiceAccessResponse). There is no need to define a variable currentServiceAccessInformation, you can use resourcedirectly.

We are maintaining multiple concurrent sessions in the Media Session Handler. This means that multiple applications can be connected to the same Media Session Handler (see handling in handleServiceAccessResponse). There is no need to define a variable currentServiceAccessInformation, you can use resourcedirectly.

Because in some Consumption reporting related methods such as IsConsumptionReportingActivated and NeedReportConsumption, clientConsumptionReportingConfiguration of ServiceAccessInformation is need. So I need to save it as a member of the class MediaSessionHandlerMessengerService. I can only use resource directly only in handleStartPlaybackByServiceListEntryMessage.

That is what clientsSessionData can be used for:

clientsSessionData[sendingUid]?.serviceAccessInformation

Got it, thanks. Done

dsilhavy · 2023-09-04T06:53:02Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

+                    if(currentServiceAccessInformation!!.clientConsumptionReportingConfiguration.serverAddresses.isNotEmpty())
+                    {
+                        val serverAddressesForConsumpReport: String  = currentServiceAccessInformation!!.clientConsumptionReportingConfiguration.serverAddresses[0];
+                        Log.i(TAG, ">>>shilin: clientConsumptionReportingConfiguration serverAddresses0:  $serverAddressesForConsumpReport.")


Remove comment

dsilhavy · 2023-09-04T06:56:41Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

+    }
+
+    private fun initializeRetrofitForConsumpReport(url: String) {
+        val m5RetrofitConsump: Retrofit = Retrofit.Builder()


retrofitBuilder is already initialized you can use

val retrofit = retrofitBuilder .baseUrl(m5BaseUrl) .build()

dsilhavy · 2023-09-04T06:58:02Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

+            .baseUrl(url)
+            //.addConverterFactory(GsonConverterFactory.create())
+            .build()
+        consumptionReportingApi     = m5RetrofitConsump.create(ConsumptionReportingApi::class.java)


This needs to be bound to a specific client is as there are multiple applications connected to the same Media Session Handler. As an example for the Service Access Information:

clientsSessionData[msg.sendingUid]?.serviceAccessInformationApi = retrofit.create(ServiceAccessInformationApi::class.java)

dsilhavy · 2023-09-04T06:58:25Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

@@ -144,6 +169,7 @@ class MediaSessionHandlerMessengerService() : Service() {
                }

                override fun onFailure(call: Call<ServiceAccessInformation?>, t: Throwable) {
+                    Log.i(TAG, ">>>shilin: debug onFailure")


Remove comment

dsilhavy · 2023-09-04T07:01:52Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

+            return false;
+        }
+
+        if(SamplePercentage_100.toUInt() == samplePercentage)


Use this function to check if reporting should be activated or not according to samplePercentage:

private fun shouldReportAccordingToSamplePercentage(samplePercentage: Float?): Boolean { if (samplePercentage != null && samplePercentage <= 0) { return false } if (samplePercentage == null || samplePercentage >= 100.0) { return true } return utils.generateRandomFloat() < samplePercentage }

dsilhavy · 2023-09-04T07:05:18Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

+
+        // check clientConsumptionReportingConfiguration.locationReporting
+        // check clientConsumptionReportingConfiguration.accessReporting
+        if(currentServiceAccessInformation!!.clientConsumptionReportingConfiguration.locationReporting


Why is consumption reporting only send if locationReporting or accessReporting is true?

Just follow clause 4.7.4, as below:
If the consumption reporting procedure is activated, the Media Session Handler shall submit a consumption report to the 5GMSd AF when any of the following conditions occur:

Start of consumption of a downlink streaming session;

Stop of consumption of a downlink streaming session;

Upon determining the need to report ongoing 5GMS consumption at periodic intervals determined by the clientConsumptionReportingConfiguration.reportingInterval property.
- Upon determining a location change, if the clientConsumptionReportingConfiguration.locationReporting property is set to True.
- Upon determining an access network change (e.g. unicast to eMBMS, or vice versa), if the clientConsumptionReportingConfiguration.accessReporting property is set to True.

Did I misunderstand it?

dsilhavy · 2023-09-04T07:06:00Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

+    private fun reportConsumption(msg: Message) {
+        if (!NeedReportConsumption())
+        {
+            Log.i(TAG, "shilin>>>Not need ReportConsumption")


Remove all shilin from the logs

dsilhavy · 2023-09-04T07:06:07Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

+                response: Response<ResponseBody?>
+            ) {
+                Log.i(TAG, "shilin>>resp from AF:"+ response.body()?.string())
+                //System.out.println(">>>>>>>>>>>>"+response.body());


Remove comment

dsilhavy · 2023-09-04T07:06:18Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/network/ConsumptionReportingApi.kt

+    @FormUrlEncoded
+    @POST("consumption-reporting/{aspId}")
+    fun postConsumptionReporting(@Path("aspId") aspId: String?, @Field("data") data: String?): Call<ResponseBody>?
+    //fun postConsumptionReporting(@Path("aspId") aspId: String?, @Field("data") data: String ): Call<String>?


Remove comment

dsilhavy · 2023-09-04T07:07:57Z

Another general comment: The Media Session Handler maintains a timer to request consumption reports from the Media Stream Handler. This seems to be missing here?

…ngConfiguration.samplePercentage from UInt to Float

shilinding · 2023-09-07T08:01:25Z

Another general comment: The Media Session Handler maintains a timer to request consumption reports from the Media Stream Handler. This seems to be missing here?

The current implementation is: Media Stream Handler maintains a timer to report consumption reports to Media Session Handler, the timer is in Media Stream Handler, is it OK?

dsilhavy · 2023-09-07T08:47:11Z

Another general comment: The Media Session Handler maintains a timer to request consumption reports from the Media Stream Handler. This seems to be missing here?

The current implementation is: Media Stream Handler maintains a timer to report consumption reports to Media Session Handler, the timer is in Media Stream Handler, is it OK?

In my opinion, the Media Session Handler should maintain the timer according to reportingInterval. This would match with what we are doing for metrics reporting. If the reportingInterval changes during a streaming session the Media Session Handler can act accordingly and adjust its timer.

However, your approach might also be valid if I look at 26.501 5.6.1. But you need to make sure that you report according to reportingInterval.It looks like this is still a Todo here: https://github.com/5G-MAG/rt-5gms-media-stream-handler/pull/49/files

We can discuss the best approach tomorrow in the call. Maybe @rjb1000 also has some thoughts on this.

rjb1000 · 2023-09-07T11:53:51Z

In my opinion, the Media Session Handler should maintain the timer according to reportingInterval. This would match with what we are doing for metrics reporting. If the reportingInterval changes during a streaming session the Media Session Handler can act accordingly and adjust its timer.

However, your approach might also be valid if I look at 26.501 5.6.1. But you need to make sure that you report according to reportingInterval. It looks like this is still a Todo here: https://github.com/5G-MAG/rt-5gms-media-stream-handler/pull/49/files

We can discuss the best approach tomorrow in the call. Maybe @rjb1000 also has some thoughts on this.

TS 26.501 probably doesn't specify whether the reporting interval timer should be in the Media Session Handler or the Media Stream Handler. What is important is that the Media Session Handler regularly checks for changes to the client metrics reporting configuration by re-requesting the Service Access Information and that the 5GMS Client takes account of any changes in reporting frequency.

I couldn't find many clues in TS 26.512 clauses 12 and 13.

Clause 13.2.4 defines a metricsConfiguration object exposed by the Media Session Handler (Media Player) at reference point M8. But there is no equivalent object for the consumption reporting configuration. This looks like a specification gap. Please raise a Standards issue to record this.

Table 13.2.5-1 defines some metrics-related notification events sent by the Media Stream Handler (Media Player) to the Media Session Handler at reference point M7:

METRIC_ADDED: Triggered every time a new metric is added.

METRIC_CHANGED: The minimum bit rate that the ABR algorithms will choose. Use NaN for no limit.

METRIC_UPDATED: Set to true if you would like DASH Client to keep downloading fragments in the background when the video element is paused.

METRICS_CHANGED: Triggered whenever there is a change to the overall metrics.

There are no notifications defined for consumption reporting. Again, this could be a specification gap that needs reporting as a Standards issue.

Given these specification gaps, it seems that this is all up to implementation choice. That said, it would make sense for the consumption reporting feature and the QoE metrics reporting feature to follow a common design pattern.

In the case of QoE metrics, I would say that the Media Stream Handler (Media Player) is responsible for sampling a configured subset of metrics at a particular configured sample rate and making them available to the Media Session Handler via M7 notifications (table 13.2.5-1) and the metrics dynamic status information blob (table 13.2.6.-1).
For consumption, the Media Stream Handler (Media Player) simply logs consumption reporting units as and when something changes, and makes them available to the Media Session Handler somehow via M7. Maybe it just needs to add a consumption reporting unit to a queue (maybe needs a new property in table 13.2.6-1?) whenever there is a representation change that can be "polled" by the Media Session Handler? Or maybe a notification-based mechanism (table 13.2.5-1)? Possibly both.

The intervals at which metrics/consumption reports are sent by the Media Session Handler to the 5GMS AF feels like a concern for the Media Session Handler alone. It feels more natural for the Media Session Handler to maintain the reporting timers and to decide when to send metrics/consumption reports. (So I think I agree with @dsilhavy.) These reports would contain all information logged since the last report was successfully submitted to the 5GMS AF. (The Media Session Handler needs to handle error cases at M5 and buffer information while it retries the report submission.)

Does that make sense?

(Sorry for the long post. It should probably be turned into a discussion...)

2. refine compare between Float 3. refine log

…ssion with Metric 2. Consumption reporting support multi Clients

dsilhavy · 2023-09-12T09:31:29Z

@shilinding : I will wait for your final changes regarding the metrics reporting timer before providing another code review. Please let me know once you are done. Moreover, once the remaining changes in the Media Stream Handler are done we can run an end to end test.

shilinding · 2023-09-15T09:10:49Z

@shilinding : I will wait for your final changes regarding the metrics reporting timer before providing another code review. Please let me know once you are done. Moreover, once the remaining changes in the Media Stream Handler are done we can run an end to end test.

@dsilhavy I've push the changes including "move the timer for Consumption Report to MSH" and "move PlayerStates updating trigger to MSH", thanks.

2. print real report data

dsilhavy · 2023-09-26T08:00:08Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

+
+    fun startConsumptionReportTimer(clientId: Int) {
+        val timer = Timer()
+        clientsSessionData[clientId]?.serviceAccessInformationRequestTimer = timer


why do you overwrite the serviceAccessInformationRequestTimer? This timer is used to re-request the Service Access Information

It's a typo and it should be consumptionReportingTimer, thanks for finding the bug. Corrected

I will address the remaining issues in the changes I am currently making. As this PR is already merged please do not add any more changes at this point as they will be lost (see also my comment at the bottom)

dsilhavy · 2023-09-26T08:04:53Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

+            timer.schedule(
+                object : TimerTask() {
+                    override fun run() {
+                        clientsSessionData[clientId]?.isConsumptionReportByTimer = true


Consumption reports shall be send periodically:

Reports are sent at periodic intervals determined by the reporting interval attribute of the consumption reporting configuration specified in Table 4.2.3-2.

With this logic it seems like the next report is send only if a message from the MediaStreamHandler was dispatched and not at the predefined interval.

I feel that my understanding is not quite the same as yours, and I only found:

In 4.7.2.3: The Media Session Handler shall periodically check for updated Service Access Information

In 4.7.4: Upon determining the need to report ongoing 5GMS consumption at periodic intervals determined by the clientConsumptionReportingConfiguration.reportingInterval property.

As I understood, the report is send only if a message from the MediaStreamHandler was dispatched, and the conditions 4.7.4 occur including check the timer at the predefined interval. Am I misunderstood?

The consumption reports are sent as defined by reportingInterval. With each report the duration property is increased, see also here: 5G-MAG/rt-5gms-media-stream-handler#52

dsilhavy · 2023-09-26T08:09:42Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

+            return
+        }
+
+        val provisisioningSessionId: String = "2";


Why is this value hardcoded?

I didn't found the defintion of provisisioningSessionId in TS26.512, I didn't know how to generate it, so I hardcoded first.

Note the parameter is called provisioningSessionId, you have typos throughout your code.

Got it. Thanks

dsilhavy · 2023-09-26T08:28:27Z

@shilinding Your changes are a good starting point. I noticed that there are still quite a few hardcoded values in the Media Stream Handler and the Media Session Handler. Moreover, at least one change in this PR breaks existing functionality not related to consumption reporting.

As I don't have access to the underlying branch for this PR I will merge your changes to development and then work myself on fixing the remaining issues in a new branch.

dsilhavy · 2023-09-26T11:12:15Z

app/src/main/java/com/fivegmag/a5gmsmediasessionhandler/MediaSessionHandlerMessengerService.kt

+            return true;
+        }
+
+        // if the generated random number is of a lower value than the samplePercentage value


This seems to be re-evaluated for each consumption report while it should only be re-evaluated for each new streaming session

As I understand, it's purpose is to control the percentage of report(to reduce network footprint?)
I think may the description of this part of the spec is very unclear.@rjb1000

Yes, you're right, @shilinding. When less than 100% of 5GMS Clients send consumption reports, the load on the 5GMS AF is reduced.

TS 26.512 clause 11.2.3.1 defines ServiceAccessInformaion.clientConsumptionReportingConfiguration.samplePercentage as:

The percentage of media streaming sessions that shall send consumption reports, expressed as a floating point value between 0.0 and 100.0.

When we discussed how to implement this aspect of metrics reporting, we realised that there is an implementation choice about exactly when to evaluate the random number generator that drive this behaviour:

Generate the random number once for each streaming session. Reporting is then either enabled or disabled for that streaming session.

Generate the random number each time you are thinking about sending a report.

This is left ambiguous in TS 26.512, and I think that's fine because it is a case where we can leave it to implementation choice and get a similar server load result across an entire population of 5GMS Clients.

For metrics reporting, we chose option 1. I think @dsilhavy's comment is suggesting that we choose option 1 for consumption reporting as well for consistency.

For option 1, there remains an open question in my mind about the implementation. If the random number determines that reporting is disabled for a session, does the 5GMS Client still need to collect samples, just in case the reporting configuration changes? Should our 5GMS Client implementation re-evaluate the random number if the client configuration changes? And, if it then needs to start reporting, does it need to report historic data? (Maybe that applies more to metrics reports than consumption reports, though.)

Probably related to the issue created here: #32

shilinding · 2023-09-27T10:54:06Z

@shilinding Your changes are a good starting point. I noticed that there are still quite a few hardcoded values in the Media Stream Handler and the Media Session Handler. Moreover, at least one change in this PR breaks existing functionality not related to consumption reporting.

As I don't have access to the underlying branch for this PR I will merge your changes to development and then work myself on fixing the remaining issues in a new branch.

I've applied for access to my forks for you and waiting for the Opensource team's approval. If done I'll let you know and won't add any more changes at this point untill you done. Thanks.

Shilin Ding and others added 3 commits August 25, 2023 23:05

test first commit

16332c8

Merge branch '5G-MAG:development' into development

5468ef3

get Consumption report from Media Stream Handler and post to AF via m5

0b2ee54

dsilhavy self-requested a review September 4, 2023 06:02

dsilhavy added this to the Version 1.0.3 milestone Sep 4, 2023

dsilhavy requested changes Sep 4, 2023

View reviewed changes

rjb1000 changed the title ~~Development~~ 5GMS Consumption reporting: Add support to Media Session Handler Sep 6, 2023

change data type of ServiceAccessInformation.clientConsumptionReporti…

fcb38c6

…ngConfiguration.samplePercentage from UInt to Float

Shilin Ding added 4 commits September 7, 2023 20:26

1. reuse retrofitBuilder with Metric Reporting

306a7dd

2. refine compare between Float 3. refine log

1. use the same clientsSessionData currentServiceAccessInformation se…

8f1190f

…ssion with Metric 2. Consumption reporting support multi Clients

small code optimizations

94b055f

replace parameter aspId by provisisioningSessionId

13898d3

shilinding and others added 3 commits September 14, 2023 17:19

Merge branch '5G-MAG:development' into development

7e40dc9

move the timer for Consumption Report to MSH, in line with Metric Report

87d9c2b

move PlayerStates updating trigger to MSH

00daeec

shilin.ding added 2 commits September 20, 2023 16:04

1. rename isConsumptionReport to isConsumptionReportByTimer

66c8df7

2. print real report data

Support get real IP addr in MSH

0b94262

dsilhavy modified the milestones: Version 1.0.3, Version 1.1.0 Sep 26, 2023

dsilhavy reviewed Sep 26, 2023

View reviewed changes

dsilhavy mentioned this pull request Sep 26, 2023

5GMS Consumption reporting: Pass fake consumption data to Media Session Handler 5G-MAG/rt-5gms-media-stream-handler#49

Merged

dsilhavy merged commit 4335c7e into 5G-MAG:development Sep 26, 2023
1 check failed

dsilhavy reviewed Sep 26, 2023

View reviewed changes

shilinding self-assigned this Nov 27, 2023

5GMS Consumption reporting: Add support to Media Session Handler #38

5GMS Consumption reporting: Add support to Media Session Handler #38

Conversation

shilinding commented Sep 1, 2023 • edited by rjb1000 Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dsilhavy commented Sep 4, 2023

shilinding commented Sep 7, 2023

dsilhavy commented Sep 7, 2023

rjb1000 commented Sep 7, 2023 • edited Loading

dsilhavy commented Sep 12, 2023

shilinding commented Sep 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dsilhavy commented Sep 26, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shilinding commented Sep 27, 2023

shilinding commented Sep 1, 2023 •

edited by rjb1000

Loading

rjb1000 commented Sep 7, 2023 •

edited

Loading