SonarPython 4.24.0.18631

Release notes - SonarPython - 4.24

Bug

SONARPY-2209 Avoid duplicating re-exported stub symbols when deserializing stubs

SONARPY-2216 Type of aliased ImportFrom is not propagated

SONARPY-2249 Fix descriptor to symbol conversion when an ambiguous descriptor has multiple candidates having the same fully qualified name

SONARPY-2276 Fix FP on S930 when an object method assigned to a variable

New Feature

SONARPY-2201 Introduce a type checker to check a type full name

SONARPY-2203 Migrate S2201 IgnoredPureOperationsCheck to the new type model

SONARPY-2204 Migrate S5795 IdentityComparisonWithCachedTypesCheck to the new type model

SONARPY-2207 Migrate S6740 PandasReadNoDataTypeCheck to the new type model

Sub-task

SONARPY-2256 Resolve fully qualified names for class members descriptors

False Positive

SONARPY-2229 Fix FP on S2201 when instantiating the result of a "type" call

SONARPY-2255 Aliased class types shouldn't create FPs on S5756

SONARPY-2284 Fix FP on S2638 when an imported class has a decorator on a function that is overriden in sub classes.

Task

SONARPY-1772 Represent decorators in FunctionType

SONARPY-2194 Prevent the creation of ObjectType[UnknownType] when converting VariableDescriptor

SONARPY-2199 Remove usage of V1 types in V2 type inference visitor

SONARPY-2200 Rename LazyType#fullyQualifiedName to LazyType#importPath

SONARPY-2296 Fix quality issues: ensure calls to Name#symbolV2 are checked for null

SONARPY-2326 Add Version 3.13 as supported version of Python

SONARPY-2336 Ensure ClassDescriptor and FunctionDescriptor fullyQualifiedName are not nullable

SONARPY-2340 Add tests for Python 3.13 showing error-free parsing

SONARPY-2355 Update license to SONAR Source-Available License v1.0 (SSALv1)

Improvement

SONARPY-2110 Make the ProjectLevelSymbolTable return the Descriptor model instead of the v1 Symbol model

SONARPY-2111 Implement the creation of Descriptor models out of a file's SymbolV2 / PythonType state map

SONARPY-2187 Extract the SymbolV2 / PythonType state map at the end of a module execution

SONARPY-2188 Introduce the concept of UnresolvedImportType

SONARPY-2189 Represent Django views in the updated project level symbol table

SONARPY-2191 Integrate class definition statements in the Python CFG

SONARPY-2192 Extract the SymbolV2 / PythonType state map in presence of try/except

SONARPY-2215 Extract the SymbolV2 / PythonType state map when the cfg is null

SONARPY-2250 Need to resolve a type when it is pointing to nested class

SonarPython 4.23.0.17664

Release notes - SonarPython - 4.23

Bug

SONARPY-1572 Fix parse error for challenging unicode characters

SONARPY-1573 Fix parse error of unparenthesized walrus in set literals and comprehension

SONARPY-1727 S5905: Make sure the quick fix removes trailing commas

SONARPY-1792 Cannot parse a file with binary characters that is valid in the Python interpreter

SONARPY-2068 Tabulations (\t) should be handled correctly as an escape character

SONARPY-2081 Array index in hex form should be handled correctly

SONARPY-2120 Ensure coverage report sensor fails gracefully when reports can't be read

SONARPY-2196 Fix off-by-one errors in the highlighting/issue location in Jupyter Notebooks

Documentation

SONARPY-2005 S1721 : Update rule for in (...)

SONARPY-2015 Fix S5654 RSPEC broken link

SONARPY-2016 Make rule examples for S6738 and S6711 module-level

SONARPY-2195 S2068: Update title to "Hard-coded passwords are security-sensitive"

False Negative

SONARPY-2153 S6978: Fix False Negative related to resolving `torch.nn.Module`

New Feature

SONARPY-1995 Support Jupyter Delimiter in Jupyter Notebooks lexing

SONARPY-2132 Support Union types in SymbolsModuleTypeProvider#getReturnTypeFromSymbol

Sub-task

SONARPY-2168 FunctionDescriptorToPythonTypeConverter should set STUB type origin

False Positive

SONARPY-633 FP on rule S5719 when abc.abstractstaticmethod is used

SONARPY-862 S117 (LocalVariableAndParameterNameConventionCheck) shouldn't raise on type aliases

SONARPY-987 S1542 / FunctionNameCheck should not raise on unittest functions 'setUpModule' and 'tearDownModule'

SONARPY-1002 Fix FP on S2275 for nested replacement fields with format specifiers

SONARPY-1018 Fix S5727 to not raise when comparing xml ElementTree.getroot() to None

SONARPY-1531 Verify that private builtins are not serialized.

SONARPY-1711 Rule S5659: do not raise under certain use of get_unverified_header()

SONARPY-1756 FP on rule S5806 when the function name is used

SONARPY-1814 S1481: Fix detection of mutation of dict using the |= operator

SONARPY-1834 S6969 triggers when a Pipeline is used in a ColumnTransformer

SONARPY-1974 S1481: should not raise on an assignment expression occurring in a generator expression

SONARPY-2038 S4790 support usedforsecurity=False in python

SONARPY-2182 Fix FP on S5756 when calling TypedDict

SONARPY-2183 Fix FP on S5708 when the caught object has type "type"

SONARPY-2184 S125: Fix FP when analyzing databricks notebook

Task

SONARPY-1022 Change PythonCustomRuleRepository#checkClasses return type to List<Class<?>>

SONARPY-1913 Update Microsoft stubs

SONARPY-2094 Refactor Python version comparison to PythonVersionsUtils

SONARPY-2095 Update pluginApiMinVersion in pom.xml

SONARPY-2097 S6971: Fix incomplete quickfix message

SONARPY-2164 Update sonar-plugin-api to version 10.11.0.2468

SONARPY-2169 Update logback dependency to 1.5.8

SONARPY-2181 Ensure UnionType can never contain LazyType

SONARPY-2193 Update sonarlint-core.version to v10 (major)

SONARPY-2197 Update protobuf.version to v4 (major)

SONARPY-2198 Update mockito.version to v5 (major)

Improvement

SONARPY-1750 S6928 : Allow calls to python functions that have the decorator @tf.py_function

SONARPY-2084 S1451 : Improve issue message

SONARPY-2114 Migrate S5707 ExceptionCauseTypeCheck to the V2 type model

SONARPY-2115 Migrate S1244 FloatingPointEqualityCheck to the V2 type model

SONARPY-2161 Replace v2 TypeShed with TypeShedDescriptorsProvider

SONARPY-2163 Migrate S5708 CaughtExceptionsCheck to the V2 type model

SONARPY-2170 SONARPY-2170 Support detailed return types in Descriptor model

SonarPython 4.22.0.16914

Release notes - SonarPython - 4.22

Bug

SONARPY-2087 Formatted raw strings with double backslashes should be parsed correctly

SONARPY-2136 Remove microsoft_stub test

SONARPY-2147 Fix Message of Rule S6973

Documentation

SONARPY-2077 S2068: Update documentation to not recommend the customization

SONARPY-2078 S6418: Create RSPEC S6418 for Python (this rule already exists for Java)

False Negative

SONARPY-2151 S6982: Fix fn when import torch is used instead of the full import name torch.nn

SONARPY-2152 Lower entropy for S6418

New Feature

SONARPY-1513 Rule S6795: Generic type statement should not use TypeVars

SONARPY-1898 Rule S6978 : Subclasses of "torch.nn.Module" should call the initializer

SONARPY-1902 Rule S6979: "torch.tensor" should be used instead of "torch.autograd.Variable"

SONARPY-1907 Rule S6983 : The nb_workers parameter should be specified for torch.utils.data.DataLoader

SONARPY-1909 Rule S6984: Einops pattern should be valid

Sub-task

SONARPY-2150 Fix FP on S6982 when method is used on optimizers

Task

SONARPY-1337 Use sonar-plugin-api to fetch input file hashes

SONARPY-1588 Deprecate rule S4792

SONARPY-1899 Add relevant projects to Peach

SONARPY-2080 S6418: Move ShannonEntropy to analyzer commons

SONARPY-2083 Fix parsing error when encountering `#@title` in a notebook cell

SONARPY-2140 Add containsSpreadOperator to Expresions utils class

SONARPY-2148 Add pytorch and torchvision custom stubs

Improvement

SONARPY-1900 Rule S6973: Implement PyTorch Hyperparameter detection

SONARPY-1910 Rule S6982: module mode should be set after load_state_dict

SONARPY-1914 Rule S6985: Usage of "torch.load" can lead to untrusted code execution

SONARPY-1915 Expand S6929: add Pytorch and quickfix

SONARPY-2079 S6418: Implementation for Python

SONARPY-2143 S6985: Check if spreaded arguments are present in the "torch.load" call

SonarPython 4.21.0.16473

Release notes - SonarPython - 4.21

Bug

SONARPY-2019 Fix NPE for workDir in IPynbSensorTest

SONARPY-2025 PythonScanner parses the wrong file when given a GeneratedFile

SONARPY-2027 IPynbParser should create new lines when parsing multiline strings code cells

SONARPY-2045 Fix incorrect count of escaped chars

SONARPY-2046 Fix missing enrichment of tokens

SONARPY-2049 Remove extra line break when the last line of a source array ends with \n

SONARPY-2051 Single quotes should not count as escaped characters in TokenEnricher or IPynbNotebookParser

SONARPY-2057 Markdown content should not leak in the generated file

SONARPY-2058 Fix offset computation on single line JSON files

SONARPY-2061 Fix bug when no code is present in the notebook

SONARPY-2063 Multiline strings should have a correct end position

SONARPY-2066 IPython help command should be parsed correctly

SONARPY-2067 First line of the first cell of notebooks should end at the correct location

SONARPY-2076 Fix highlighting of single line Jupyter Notebook

False-Positive

SONARPY-2053 Investigate FP on S6999

SONARPY-2056 IPython line magics starting with numbers or special characters should be parsed correctly

New Feature

SONARPY-1983 Implement IPythonNotebook parser

SONARPY-1986 The IPythonSensor should register to .ipynb files

SONARPY-2018 Only analyze notebooks with a Python kernel

Task

SONARPY-1987 IPynbSensor should not crash when an error occurs during the parsing of notebooks

SONARPY-1991 Adapt metrics computation for IPython files

SONARPY-1992 Add Jupyter notebooks to our Ruling tests

SONARPY-1993 Add plugin tests for the analysis of notebooks

SONARPY-1994 Add projects with notebooks to Peach

SONARPY-2002 Implement basic notebook parser with support for array code cells

SONARPY-2003 Implement basic notebook parser with support for multiline string code cells

SONARPY-2004 Track escaped characters correctly in notebook parser

SONARPY-2006 Verify the parsing of notebooks works with the whole json on a single line.

SONARPY-2012 IPynbParser should return a GeneratedIPythonFile

SONARPY-2032 Adapt (executable) lines of code metric for Jupyter Notebooks

SONARPY-2033 Adapt Highlighter for Jupyter Notebooks

SONARPY-2035 Update TokenLocation to make use of the correct location for reporting

SONARPY-2040 Add the sonar.ipynb.file.suffixes property

SONARPY-2041 Implement orchestrator cache

SONARPY-2042 Feed IPythonTreeMaker with GeneratedIPythonFile data

SONARPY-2043 Deactivate CPD for Notebooks

SONARPY-2059 PythonScanner should log the parsing exception on the correct line for notebooks

Improvement

SONARPY-1988 Create an interface to wrap InputFile

SONARPY-1989 Extend Token to hold original and virtual locations

SONARPY-1990 Map JSON locations to python locations in IPythonTreeMaker

SONARPY-2001 Create GeneratedIPythonFile to hold offset data

SONARPY-2036 Enrich EOF tokens

SONARPY-2047 Enrich trivia tokens

SonarPython 4.20.0.16410

Release notes - SonarPython - 4.20

Bug

SONARPY-2055 Analysis should stop for notebooks after a parse error

SONARPY-2069 Fix usage of V1 TypeShed during resolution of V2 types

SONARPY-2070 Fix NPE when type inference visit is missing

New Feature

SONARPY-2044 Add STIG metadata support

Task

SONARPY-486 Update assertions on issue locations in unit tests

SONARPY-1591 Update Python protobuf version to 4.25.1

SONARPY-1917 Investigate decoupling class/function information from types

SONARPY-1918 Investigate representation of project table in type model V2

SONARPY-1920 Investigate how to represent declared types (type sources)

SONARPY-1921 Investigate next rules to migrate

SONARPY-1929 Specify type checking APIs to work with non-exact type sources

SONARPY-1930 Specify how to represent overloads in the type model

SONARPY-1932 Specify how to represent Type Aliases

SONARPY-1943 Investigate TypeShed build inside the docker container

SONARPY-1945 CI pipeline should fail if there are differences in checksum for typeshed stub generation.

SONARPY-1957 Explore JSON parsing solutions

SONARPY-1958 Explore the possibility of analyzing raw files in sonar-python

SONARPY-1959 Explore current support of Jupyter Notebooks

SONARPY-1960 Investigate issue reporting APIs to ensure precise locations are always raised on ipynb files

SONARPY-1965 Investigate raw parsing of Notebooks with SSLR

SONARPY-1970 Investigate String escaping issue with Highlighting, issue reporting

SONARPY-1972 Investigate where remapping location should take place

SONARPY-1973 Investigate feasibility of ipynb analysis in IntelliJ sonar lint

SONARPY-2034 Refactor IPythonLocation to hold an ordered list rather than a linkedHashMap

SONARPY-2071 Update parent pom to 75.0.0.1943

Improvement

SONARPY-1480 Optimization of plugin download

SONARPY-1934 Migrate rule S3699 UseOfEmptyReturnValue to typeV2

SONARPY-1935 Migrate rule S112 GenericExceptionRaised to typeV2

SONARPY-1936 Migrate rule S5864 NonCallableCalled to typeV2

SONARPY-1953 Set type source for function return type

SONARPY-1984 Infer type of function parameters based on its type hints

SONARPY-1985 Infer a function return type based on its type hint for locally defined functions

SONARPY-1996 Implement a basic TypeChecker builder

SONARPY-1997 Propagate function return types to the result of call expressions

SONARPY-2000 Propagate types of imported names

SONARPY-2010 Avoid creating duplicate types when the corresponding symbols are duplicated

SONARPY-2017 Avoid type related FP in case of isInstance checks

SONARPY-2022 Resolve type aliases types from TypeShed

SONARPY-2023 Translate type source for binary expressions

SONARPY-2024 Ensure call expressions return types have same type source as their callee

SONARPY-2029 Implement API to get a module type's member type from project level types table

SONARPY-2064 Propagate imported names from import from

SonarPython 4.19.0.15616

Release notes - SonarPython - 4.19

Bug

SONARPY-1846 Fix analysis warnings not being concatenated correctly with newline delimiters

False-Positive

SONARPY-1518 Avoid relying on stubs from a library if it is the project under analysis

Task

SONARPY-1758 Create basic flow insensitive type inference engine v2

SONARPY-1765 Infer type of qualified expressions

SONARPY-1796 Infer types for set, dict and tuple literals

SONARPY-1797 Introduce new type inference model classes

SONARPY-1798 Try to resolve built-in types for names which have no symbol

SONARPY-1800 NonCallableCallCheck migration: Use the display name of the new type model

SONARPY-1803 Store type definition location in the new type model

SONARPY-1807 Populate symbol table out of SymbolTableBuilder and make it accessible from TypeInferenceV2

SONARPY-1808 Implement API to get symbols of global vars of the module

SONARPY-1809 Implement API to get symbols of local vars of the given function definition

SONARPY-1810 Track types in case of multiple assignments in module scope

SONARPY-1815 Enable AST-based type inference for functions/module containing try/catch blocks

SONARPY-1816 Ensure type inference doesn't lead to FPs in presence of isinstance checks for runtime types

SONARPY-1817 Ensure member access types are resolved correctly

SONARPY-1818 Enable flow sensitive type inference within functions

SONARPY-1819 Improve testing to compare types

SONARPY-1824 Fix incorrect inference for global variables assigned within functions

SONARPY-1825 Return Python.UNKNOWN instead of ObjectType[PythonType.UNKNOWN] for unknown call expressions

SONARPY-1826 Enable flow sensitive type inference for function types

SONARPY-1830 Prevent single-assigned module scope types from being propagated to nested functions

SONARPY-1836 Infer item type for basic loop over list iteration

SONARPY-1866 Infer types of reassigned parameters even when they don't have type annotations

Improvement

SONARPY-1782 Migrate S5756 NonCallableCalledCheck to the new type model

SonarPython 4.18.0.15334

Release notes - SonarPython - 4.18

False-Positive

SONARPY-1777 S1128: Do not raise an unused import on from sklearn.experimental ...

New Feature

SONARPY-1768 Rule S6969 : "memory" parameter should be specified for Scikit-Learn Pipeline

SONARPY-1770 Rule S6971 : Transformers should not be accessed directly when a Scikit-Learn Pipeline uses caching

SONARPY-1771 Rule S6972: Nested estimator parameters adjustment in a Pipeline should refer to valid parameters

SONARPY-1775 Rule S6973 : Important hyperparameters should be specified for Scikit-Learn estimators

SONARPY-1780 Rule S6974: Subclasses of Scikit-Learn's "BaseEstimator" should not set attributes ending with "_" in the "__init__" method

Improvement

SONARPY-1769 Modify S6709: To include Scikit learn

SonarPython 4.17.0.14845

Release notes - SonarPython - 4.17

Bug

SONARPY-1732 Fix non-determinism on S2638 when positional-only parameters have no name available

New Feature

SONARPY-1647 S6929: The axis argument should be specified when using reduction operations

SONARPY-1648 S6928: Python side effects should not be used inside a tf.function

SONARPY-1649 S6908: tf.function should not be recursive

SONARPY-1650 S6911: tf.function should not depend on global or free python var

SONARPY-1652 S6918: "tf.Variable" objects should be singletons when created inside of a "tf.function"

SONARPY-1654 S6919: The input_shape parameter should not be specified for tf.keras.Model subclasses

SONARPY-1657 S6925: The validate_indices argument should not be set for tf.gather function call

Improvement

SONARPY-1686 Define sonar.python.version as a public property

SonarPython 4.16.0.14672

Release notes - SonarPython - 4.16

False-Positive

SONARPY-1685 Fix FP on S6742 when DataFrame class is not a pandas's library class

New Feature

SONARPY-1636 S6903: Using timezone aware "datetime"s should be preferred over using "datetime.datetime.utcnow" and "datetime.datetime.utcfromtimestamp"

SONARPY-1637 S6894: A date should be formatted correctly when using "pandas.to_datetime" with "dayfirst" or "yearfirst" arguments

SONARPY-1639 S6882: Constructor attributes of date and time objects should be in the range of possible values

SONARPY-1640 S6883: The 12-hour format should be used with the AM/PM marker, otherwise 24-hour format should be used

SONARPY-1643 S6900: Numpy weekmask should have a valid value

SONARPY-1644 Rule S6890: zoneinfo should be preferred to pytz when using Python 3.9 and later

SONARPY-1645 Rule S6887: pytz.timezone should not be passed to the datetime.datetime constructor

SonarPython 4.15.0.14551

Release notes - SonarPython - 4.15

Bug

SONARPY-1593 Add support for star symbol in type annotation

SONARPY-1635 OneStatementPerLine: do not raise an issue on dummy function and class implementations

False-Positive

SONARPY-1598 Fix FP on S6542 with re-exported/imported override

SONARPY-1600 Fix FP on S5655: boolean parameter incompatible with float type annotation.

SONARPY-1609 Fix FP on S5886 when returning optional unions of unknown symbols

SONARPY-1612 S905: should ignore assigned and unused Airflow operators.

SONARPY-1613 S905: should ignore __manifest__.py files

SONARPY-1617 S117: Ignore variable assigned to "namedtuple" or "NamedTuple"

SONARPY-1622 Fix FP on S5886 when returning collections.abc.AsyncIterable

SONARPY-1624 Fix FP on S5886 when the expected return type is a tuple subclass

SONARPY-1628 S1172: should not raise on methods of abstract classes

SONARPY-1629 Prevent S5756 from being raised on calls to instances of NamedTuple

SONARPY-1630 S1192: Ensure no issue is raised on test code

SONARPY-1660 Fix FP on S905 when using a decorator to declare an Airflow DAG

SONARPY-1662 Fix FP on S117 when a class is assigned to a variable

Improvement

SONARPY-1633 S1481: Add a quick fix to rename loop indices

SONARPY-1634 S1481: Add a quick fix to remove assignment target