Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Can't flush stdout at shutdown #1505

Open
Halpph opened this issue Jun 7, 2024 · 1 comment
Open

[Bug] Can't flush stdout at shutdown #1505

Halpph opened this issue Jun 7, 2024 · 1 comment
Assignees

Comments

@Halpph
Copy link

Halpph commented Jun 7, 2024

Expected Behavior

Hello everyone, I have some azure functions that were working for long time and now they stopped, also locally they work fine, so I tried to investigate for the error, I think the error is due to the fact that I created an AzureLogHandler, that's supposed to log some informations to my Azure Log Analytics and the flush is not well supported. I see that your code is calling sys.stdout.flush() here but I guess that's not the right way, maybe it should be done calling logging.shutdown(), so that it doesn't fail.

Actual Behavior

The Azure function is running fine, but at the end I get this error: Result: Failure Exception: AttributeError: 'PrintLogger' object has no attribute 'flush' Stack: File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 643, in _handle__invocation_request sys.stdout.flush()
I'd like it to stop throwing this error.

Steps to Reproduce

No response

Relevant code being tried

from collections import namedtuple
import json
import os
import datetime
from laura.powerbi_pilot import PowerBIPilot
import logging
import azure.functions as func
from opencensus.ext.azure.log_exporter import AzureLogHandler


def get_powerbi_vars(workspace_id):
    powerbi_vars = namedtuple("powerbi_vars", [
        'appid',
        'password',
        'tenant_id',
        'workspace_id'
    ])

    t = [os.environ['PowerBiSPAppId'],
         os.environ['PowerBiSPSecretId'],
         os.environ['TenantId'],
         workspace_id
         ]

    powerbi_vars = powerbi_vars._make(t)
    return powerbi_vars


def main(logdatasetrefreshtimer: func.TimerRequest) -> None:
    utc_timestamp = datetime.datetime.utcnow().replace(
        tzinfo=datetime.timezone.utc).isoformat()

    if logdatasetrefreshtimer.past_due:
        logging.info('The timer is past due!')

    data_logger = logging.getLogger("data_logging")
    data_logger.setLevel(logging.INFO)

    logging.info('Fetching APPINSIGHTS key')
    app_insights_instr_key = os.environ["APPINSIGHTS_INSTRUMENTATIONKEY"]

    logging.info('Adding key to handler')
    data_logger.addHandler(AzureLogHandler(
        connection_string=f"InstrumentationKey={app_insights_instr_key}")
    )

    workspaces = list(os.environ["PowerBiWorkspaces"].split(","))

    for workspace in workspaces:
        logging.info(f"~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Workspace: {workspace}~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~")
        powerbi_pilot = PowerBIPilot(
            powerbi_vars=get_powerbi_vars(workspace_id=workspace))
        try:
            powerbi_pilot.get_token()
            datasets = (json.loads(powerbi_pilot.list_datasets_from_group(powerbi_pilot.workspace_id).content)['value'])
            #logging.info(datasets)
            for dataset in datasets:
                if dataset['name'] not in ["Report Usage Metrics Model","Usage Metrics Report"]:
                    logging.info(
                        f"~~~~~~~~~~~~~~ Refresh history dataset:  {dataset['id']}: {dataset['name']} ~~~~~~~~~~~~~~")
                    for refresh in (json.loads(powerbi_pilot.get_refresh_history(dataset['id']).content)['value']):
                        refresh['dataset'] = dataset['id']
                        refresh['workspace'] = workspace
                        data_logger.info(json.dumps(refresh))

        except Exception as e:
            logging.info(e)

Relevant log output

06/07/2024, 06:00:09.452
Error
Result: Failure Exception: AttributeError: 'PrintLogger' object has no attribute 'flush' Stack: File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 643, in _handle__invocation_request sys.stdout.flush()
06/07/2024, 06:00:09.452
Error
Executed 'Functions.PowerBiLogger' (Failed, Id=6799712d-8639-48e6-85c4-5a506d2a99ab, Duration=9449ms)
06/07/2024, 06:00:09.460
Error
Result: Failure Exception: AttributeError: 'PrintLogger' object has no attribute 'flush' Stack: File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 643, in _handle__invocation_request sys.stdout.flush()

requirements.txt file

azure-functions
requests
databricks-cli
opencensus-ext-azure
pandas
./domains/data_tamers/functions/shared/wheels/laura-0.22.0-py3-none-any.whl

Where are you facing this problem?

Production Environment (explain below)

Function app name

No response

Additional Information

No response

@bhagyshricompany
Copy link

@gavin-aguiar please comment and validate.Thanks

@gavin-aguiar gavin-aguiar self-assigned this Jul 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants