Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for XML file upload pre-change (DCNE-150) #77

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

adealdag
Copy link
Contributor

fixes #76

Copy link
Collaborator

@akinross akinross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@shrsr shrsr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@gmicol gmicol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@samiib samiib left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

tree = ndi.construct_tree(extract_data)
ndi.create_structured_data(tree, file)
# Check if file extension is XML. If XML, skip JSON validation/transformation
if not file.endswith(".xml"):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we base the content validation on the file extension?
I think we could achieve a nicer user experience by catching the JSON exception and checking if it is an actual XML document.

What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's another option, indeed. I am not sure how common it is for customers to use an XML file without the proper extension, but of course that would be a more solid experience. Any ideas how to check if it's a valid XML file, other than checking the content for XML tags?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think there are more ways of achieving this but I would say either lxml (https://lxml.de/validation.html) or xmlschema (https://xmlschema.readthedocs.io/en/latest/usage.html#validation)

@samiib samiib changed the title Add support for XML file upload pre-change Add support for XML file upload pre-change (DCNE-150) Aug 14, 2024
@akinross akinross force-pushed the Fix-76-nd_pcv-fails-when-file-is-XML branch from e8568bb to 26b37aa Compare October 16, 2024 08:20
@adealdag
Copy link
Contributor Author

Have modified the code according to prev conversation. Now the module is able to detect the type of file and deal with no extension or wrong extension files.

"""This method handles file download and upload operations
:arg method (str): Method can be GET or POST
:arg path (str): Path should be the resource path
:arg file (str): The absolute file path of the target file
:arg file_ext (str): The file extension, with leading dot, to be used for the file. If file has already an extension, it will be replaced
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add a check that the file extension leads with a dot? possibly error or inject when this is not done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, added

@anvitha-jain anvitha-jain requested a review from akinross October 25, 2024 09:32
plugins/httpapi/nd.py Show resolved Hide resolved
@akinross akinross self-requested a review October 28, 2024 09:21
Copy link
Collaborator

@akinross akinross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -254,6 +255,14 @@ def send_file_request(self, method, path, file=None, data=None, remote_path=None
self.method = "GET"
if method is not None:
self.method = method

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please run black -l 159 to format these changes.

try:
file_ext = ".json"
extract_data = ndi.load(open(file))
except:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
except:
except BaseException:

I think this will also fix another sanity issue.

@adealdag adealdag requested a review from samiib November 28, 2024 17:30
Copy link
Collaborator

@samiib samiib left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like black formatting needs to run again on the new changes.
Also, there appears to be one more sanity issue.

@@ -43,7 +43,7 @@
aliases: [ site ]
file:
description:
- Optional parameter if creating new pre-change analysis from file.
- Optional parameter if creating new pre-change analysis from file. XML and JSON files are supported. If no file extension is provided, the file is assumed to be JSON.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Optional parameter if creating new pre-change analysis from file. XML and JSON files are supported. If no file extension is provided, the file is assumed to be JSON.
- Optional parameter if creating new pre-change analysis from file.
- XML and JSON files are supported. If no file extension is provided, the file is assumed to be JSON.

To fix the long line sanity issue.

@akinross akinross requested a review from sajagana as a code owner December 16, 2024 10:31
@akinross akinross requested a review from samiib December 16, 2024 10:32
Copy link
Collaborator

@samiib samiib left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@shrsr shrsr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

nd_pcv fails when file is XML (DCNE-150)
7 participants