Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add BracketAwareCsvParser to parse CSV file better #494

Merged
merged 6 commits into from
Nov 22, 2024

Conversation

HashCookie
Copy link
Contributor

@HashCookie HashCookie commented Nov 19, 2024

Part of: casbin/casbin-editor#164

Original Issue
Limitations of CSV parser:

  • The csv-parse library is mainly used for processing standard CSV format.
  • Unable to handle complex policy expressions correctly
  • Improper handling of commas and parentheses within quotation marks.
  • Does not support nested expressions

Improvement plan
Custom parser:

const tokens: string[] = [];
let currentToken = '';
let inQuotes = false;
let bracketCount = 0;
  • Can handle commas inside quotation marks
  • Support nested parentheses
  • Correctly handle escape characters
image

@HashCookie
Copy link
Contributor Author

HashCookie commented Nov 20, 2024

Continue using csv-parse,improves the HelperloadPolicyLine method by modifying the parsing logic to address two significant scenarios that the previous implementation couldn’t handle effectively:

  1. Support for Nested Brackets in Policy Lines
  • In policy lines containing nested structures (e.g., keyMatch("IT", r.sub.org) or regexMatch("(abc|def)", attr)), the previous implementation would incorrectly split tokens due to its reliance on CSV parsing alone.
  • The updated logic now ensures that nested brackets are tracked and only complete expressions are added as tokens.
  1. Proper Handling of Quote Escaping
  • Policy lines with escaped quotes (e.g., keyMatch(""IT"", r.sub.org)) are now correctly processed. Escaped quotes ("") are replaced with a single quote ("), ensuring accurate policy values.

@hsluoyz hsluoyz changed the title fix: online editor gives different results to pyCasbin Enforcer.enforce feat: add BracketAwareCsvParser to parse CSV file better Nov 22, 2024
@hsluoyz hsluoyz merged commit 4c73883 into casbin:master Nov 22, 2024
9 of 10 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 22, 2024
# [5.36.0](v5.35.0...v5.36.0) (2024-11-22)

### Features

* add BracketAwareCsvParser to parse CSV file better ([#494](#494)) ([4c73883](4c73883))
Copy link

🎉 This PR is included in version 5.36.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants