Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preflight checks hackathon 2024 #2454

Open
wants to merge 55 commits into
base: develop
Choose a base branch
from

Conversation

jmcouffin
Copy link
Contributor

Hi guys,
@dosymep @sanzoghenzo @eirannejad @GiuseppeDotto @thumDer
@dnenov @davidvadkerti @dariaivanciucova
Would you give care to give me your evaluation of the work done by the few preflight checks hackathon participants?
It should not take you long, I hope to collect your input to provide the results by next monday.

  • cad_audit
  • walltypes_naming_convention
  • and radar_check

set the order and add comments following my template to evaluate:

  1. radar_check
    I like the idea, it is pretty elaborate
  2. cad_audit
    fair entry, coding style is a bit weak
  3. walltypes
    the idea is good, it lack a bit of love, and could be extended to lots of naming_convention checks on the same principes adding a UI/UX to configure the check

jmcouffin and others added 30 commits September 3, 2024 09:56
This Error has been lingering for a while, I decided to give it a shot.
Causes:
1- Document is not workshared, OR
2- The script is trying to get the .Name and .Category of document objects that doesn't own these attribute objects, this is typical for dependent elements such as sketchplanes, graphicOverrides, roomboundaries.

Solution:
Filter out element that doesn't have a name or a category and sort them under a seperate list called "Unassigned"

Give it a try.
This Model Checker swiftly verifies the extents of the Revit model. 
Placing model extents more than 10 miles from the project's 
internal origin can lead to issues with accuracy, tolerance, 
performance, and viewport display. This check ensures that the 
model remains within a 10-mile radius of the internal origin.
Fix Model Checker Attribute Error
Made the changes below per Jean-Marc's feedback.

1- you may want to present digits results in a tabular form for readability and ease in comparison.

Response: Added output.print_table

2- the project base point cannot be placed at more than 10 miles in more recent version of Revit

Response: Good to know! I removed the report for the location of the project basept.

3- you may want to use the convert unit util to display the distances in the same units as in the UI

Response: I used the unitutils to convert the internal distances to the model distances, however in metric I am having hard time to show the suffix (mm, M, km)....etc

4- you may want to limit the amount of objects output in the output windows using linkify as it may greatly slow the tool

Response: Set the limit of linkify to 10 prints per category, if there are more violating objects, then it will show a message "Manual investigation is required"

5- I tend to prefer print_md over print_html as it is more efficient and improves readability

Response: I switched most of them to use print_md. also I used pyrevit emojis instead of html colors.



please take a look and let me know if this makes sense.
Update radar_check.py - Rev1
My proposal for a comprehensive audit of cad instances. See details described in the ModelChecker class comments.
It would replace the old LISTDWGs by F. Beaupere, from the Project panel, link tools.
Bonus: It also extends the basic functionality of RPW by adding radio buttons and a cancel button.
Updated the script for better compliance and fixed issues reported by @jmcouffin and @dosymep
Update radar_check.py - Code Cleanup, expanding tabulated info
This code is based on a true story.

Naming conventions are the foundation of a project. Naming is essential for model checks, filters, and view settings. Until now, I created my checks using Revit export text files with Power-BI. PyRevit enables continuous checking of all categories against their predefined names with just a click.
Added basic rounding for coordinates for better readability,
A template for a loadable .json file.   Just a list of strings.
update extionsion.py added.   file.json template list-template added.
replace static path by select_file(*.json)
add a default path, just for having more confort
Update radar_check.py - Rounding Coordinates
walltypes_naming_convention_check
@sanzoghenzo
Copy link
Contributor

  • uses a json file just for a list of strings, it might be just a plain text file

I think here, json is more appropriate for future scalability Making this check, checking different categories for naming convention

Ok, but the current function doesn't allow to do it, it just returns a list of names.
It should return the entire dictionary loaded from the json (then it could even be moved to a pick_json function in the pyrevit library, returning None if the file is not chosen) and the access to the allowed_wall_types values should be done in the main function.

Copy link
Contributor

@sanzoghenzo sanzoghenzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still have to tackle the radar check, for that I need quite some time 😅

cadinstances = DB.FilteredElementCollector(doc).OfClass(DB.ImportInstance).WhereElementIsNotElementType().ToElements()
if cadinstances:
return cadinstances
else:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

else not needed since the if branch returns

@tay0thman
Copy link

  • some functions are too long and returns many variables; it is more or less trivial to split them into smaller, simpler functions
  • some functions are called twice or more with the same parameters, the result should be stored in a variable and reused
  • other un-pythonic things here and there, formatting needs some fixes

no time for this @tay0thman maybe you want to take the time to do this?

@jmcouffin , I'll go and start cleaning this myself, likely this weekend.

Copy link
Contributor

@sanzoghenzo sanzoghenzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm halfway through the review of the radar check, but the cognitive load of the main function is too high for me to discover other things to refactor.

I'll do a second round once the function is split into more manageable chunks

return base_point_coordinates, survey_point_coordinates


# _______________________________________________________________Get Model Units
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove

return unitsystem


# _________________________________________________ Get Design Options & Objects
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make it a docstring for the function

Comment on lines 228 to 230
# ______________________________________________________________________________
# ________________________________________________________________ MAIN FUNCTION
# ______________________________________________________________________________
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove

Comment on lines 251 to 252
output.print_md(BAD_STRG +
'Base Point is more than ' + CRITERIA_STRG)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is valid for all the other lines below: use the str.format() method instead of string concatenation

Comment on lines 280 to 284
str(convert_units(calculate_horizontal_distance(
baspt, survpt)[1], document))],
['Project Base Point to Survey Delta Y', ' ', ' ', ' ',
str(convert_units(calculate_horizontal_distance(
baspt, survpt)[2], document))],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

repeated calculation, extract the horizontal distance into a variable (or maybe that function should be 3 functions that return a single value each!)

jmcouffin and others added 16 commits November 7, 2024 10:12
renamed wall list loader to pick json
@jmcouffin
Copy link
Contributor Author

@sanzoghenzo Thank you for all that, it is a learning experience for me and will force me to review my code more carefully before submitting to the repo.
❤️ ur code reviews
Keep them coming!

@tay0thman
Copy link

@sanzoghenzo , Thank you for your time! These are tips and lessons I could never learn from a book. I’ll go ahead and make these updates.

dosymep
dosymep previously approved these changes Nov 13, 2024
Copy link
Member

@dosymep dosymep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

radar_check
Idea: Good idea to display all coordinates to check that everything is nearby.

Interface: I think all real numbers could have been rounded to the third digit.

Code: The code is hard to read, the algorithm needs more decomposition into methods and classes. Lacks PEP8 compliance. It makes sense to create a check view once to speed up the check.

cad_audit
Idea: Showing users that some files were imported and duplicated sounds interesting.

Interface: All the properties of interest are displayed in the interface, everything looks good.

Code: The code generally looks good, lacks PEP8 compliance. Cool, I didn't know that you can create an interface (FlexForm) in pyRevit this way.

walltypes
Idea: A great idea for checking models and naming compliance. It would be cool to expand this check to other categories. Add the ability to do more complex checks: contains, starts with, ends with, exact match, regular expression matches, name matches with parameter value, and others.

Interface: I recommend making a settings template file, and when a user runs a check for the first time, a default check file would be created somewhere in my documents, which he would need to configure (I need to think about the logic).
After modifying the check, I will have to make a large and complex user interface with all the settings.

Code: There is not enough code, the check is too simple. I liked that the internal pyRevit framework is used.


On average, everyone has problems with the code, it is unnecessary complexity, there is practically no use of OOP, weak compliance with pep8, there is no interface using MVVM.


In general, thanks to everyone who participated.

return ":warning: IMPORTED" # Not an external reference
exfs = cad_type.GetExternalFileReference()
status = exfs.GetLinkedFileStatus().ToString()
if status == "NotFound":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use LinkedFileStatus.NotFound and etc

extra conversion to a string worsens performance, creates an unnecessary object in memory, which will then have to be removed by the garbage collector, and also comparing a string is much slower than comparing an integer type.

""" Collect ImportInstance class from whole model or from just active view """
collector = DB.FilteredElementCollector(doc, ac_view.Id) if active_view_only else DB.FilteredElementCollector(doc)
cadinstances = collector.OfClass(DB.ImportInstance).WhereElementIsNotElementType().ToElements()
if cadinstances:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

useless check, anyway if there is an empty list, it will not be enumerated.
The rules of good form say that it is better to return an empty list instead of None

@andreasdraxl
Copy link

Thank you for the response! There are many suggestions and insights here to build on. In my entire career, no one has really talked about my code — it was always just "as long as it works!" I can already get a lot out of this constructive "critique" and am starting to understand more and more what really matters in code architecture! @dosymep @jmcouffin 🚀🚀🚀

tay0thman and others added 2 commits November 17, 2024 04:53
updates per the recent feedback by @sanzoghenzo  and @dosymep 
1- I broke the large functions into smaller functions, maintaining (one) return per function. reduced many redundancies.
2- Added docstrings in all functions
3- used str.format instead of concatentations.
4- Added rounding to 3 decimal places, added string operations to remove trailing zeros
5- reduced the amount of filteredElementCollectors in the script. attempted to use query.get_elements_by_class, didn't work for me, I have no idea how to extract objectids from a list without looping.
6- attempted to use output.insert_divider , no helpful outcome, may I need to look further more into it.
Copy link
Contributor

@sanzoghenzo sanzoghenzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still some thing that I missed, but again the cognitive load kicked in 😅

@@ -0,0 +1,624 @@
# -*- coding: utf-8 -*-
# region Imports
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry if I wasn't clear before, but regions are usually used to group concepts/logical functionality, not python primitives.

there's no value to add a region for imports (most IDEs can collapse them automatically), constants (well, maybe this could help) and "functions" in general. Use regions, for example, for "calculate distance", "collect elements" and so on

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started using them on the large functions, thank you for the tip.
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that if you find yourself using regions in functions because they are large, probably they are too large, and regions could become separate, shorter and simpler functions 😉

CRITERIA_STRG = '10 Mi (16KM) away from the Internal Origin.'
# endregion
# region Functions
def class_collectors(document=doc, class_type=None, toggle=False):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

toggle here is a bad name, and we don't really need it because we can just check if class_type is something

Comment on lines 31 to 40
if toggle:
elements = (DB.FilteredElementCollector(document).
WhereElementIsNotElementType().
OfClass(class_type).
ToElementIds())
else:
elements = (DB.FilteredElementCollector(document).
WhereElementIsNotElementType.
ToElementIds())
return elements
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if toggle:
elements = (DB.FilteredElementCollector(document).
WhereElementIsNotElementType().
OfClass(class_type).
ToElementIds())
else:
elements = (DB.FilteredElementCollector(document).
WhereElementIsNotElementType.
ToElementIds())
return elements
elements = (
DB.FilteredElementCollector(document).
WhereElementIsNotElementType()
)
if class_type:
elements = elements.OfClass(class_type)
return elements.ToElementIds()

I also see that you always call this function with a class_type, so the if branch only adds cpu time with no added value. I don't believe you'll ever need to fetch all the items of all the classes in a document, so just use

    return (
        DB.FilteredElementCollector(document)
        .OfClass(class_type)
        .WhereElementIsNotElementType()
        .ToElementIds()
    )

Comment on lines 55 to 56
three_d_view_type = ([v for v in view_3D_types
if v.ViewFamily == DB.ViewFamily.ThreeDimensional][0])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need to search for one item matching the criteria, the next function with a generator saves CPU time and memory, since it exits as soon as a match is found instead of calculating all the elements beforehand.

Suggested change
three_d_view_type = ([v for v in view_3D_types
if v.ViewFamily == DB.ViewFamily.ThreeDimensional][0])
three_d_view_type = next(
v for v in view_3D_types if v.ViewFamily == DB.ViewFamily.ThreeDimensional
)

view.SetWorksetVisibility(ws.Id, DB.WorksetVisibility.Visible)
view.IsSectionBoxActive = True
if object is not None:
view.IsolateElementTemporary(object.Id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
view.IsolateElementTemporary(object.Id)
view.IsolateElementTemporary(object.Id)

Also, you never pass an object to this function, you might as well remove these two lines and the argument from the furnction signature

basptdistance = abs(calculate_distance(baspt, INTERNAL_ORIGIN))
surveydistance = abs(calculate_distance(survpt, INTERNAL_ORIGIN))

if basptdistance > EXTENT_DISTANCE:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if basptdistance > EXTENT_DISTANCE:
if abs(basptdistance) > EXTENT_DISTANCE:

Comment on lines 374 to 405
tbdata = [['Internal Origin Coordinates',
str(INTERNAL_ORIGIN[0]),str(INTERNAL_ORIGIN[1]),str(INTERNAL_ORIGIN[2]),
' ', ' '],
['Project Base Point Coordinates to Internal Origin',
str(convert_values(baspt[0],doc)),
str(convert_values(baspt[1],doc)),
str(convert_values(baspt[2], doc)),
str(convert_units(basptdistance, document)), baseptangle],
['Survey Point Coordinates to Internal Origin',
str(convert_values(survpt[0], doc)),
str(convert_values(survpt[1], doc)),
str(convert_values(survpt[2], doc)),
str(convert_units(surveydistance, document)), surveyptangle],
['Project Base Point to Survey Delta X', ' ', ' ', ' ',
str(convert_units(calculate_delta_x(
baspt, survpt), document))],
['Project Base Point to Survey Delta Y', ' ', ' ', ' ',
str(convert_units(calculate_delta_y(
baspt, survpt), document))],
['Planar Distance between Base Point and Survey Point',
' ', ' ', ' ',
str(convert_units(calculate_horizontal_distance(
baspt, survpt), document)),
truenorthangle],
['Total Distance between Base Point and Survey Point',
' ', ' ', ' ',
str(convert_units(calculate_distance(
baspt, survpt), document)),
' '],
['Project Elevation',
' ', ' ', str(convert_values(baspt[2], doc)),
str(convert_units((survpt[2] - baspt[2]), document))]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a formatting nightmare, and could be simplified.
I personally prefer the black/ruff style formatting (if a list is too long, brackets are on their own lines and the should be one per line, indented from the brackets), but I understand that it is not the general code style inside pyrevit (because we never had one 😅 )

since the convert_unit and convert_value are always inside a str, why not move the string conversion inside those functions and drop the str here?
The second and third line are the same, it is just the point and the distance that differs; they can be turned into a function that returns the line (the angle can be calculated inside that very function)
There's still the doc / document variables problem.

Copy link

@tay0thman tay0thman Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this function below makes sense? I am getting multiple returns:

def points_distances_angles(point):
    """
    Calculates the distances and angles between the internal origin and a point.
    Args:
        point (tuple): The coordinates of the point.
    Returns:
        distances (tuple): The distances between the internal origin and the point.
        angles (tuple): The angles between the internal origin and the point.
        x (string): The unit conversion value x-coordinate of the point.
        y (string): The unit conversion value y-coordinate of the point.
        z (string): The unit conversion value z-coordinate of the point.
    """
    x, y, z = point
    x = convert_values(x)
    y = convert_values(y)
    z = convert_values(z)
    distances = convert_units(abs(calculate_distance(point, INTERNAL_ORIGIN)))
    angles = convert_units(calculate_angle(point, INTERNAL_ORIGIN))
    return x, y, z, distances, angles

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point I was making for multiple returns is not that it is forbidden, but it is useless to have multiple returns if you then use only one of the returned values, as it was done in the original script. Multiple returns are OK if they are used together by the calling function.
What I meant in this particular case, though, is this

def report_point(point, distance, document):
    angle = calculate_angle(point, INTERNAL_ORIGIN)
    return (convert_values(point[0], document), convert_values(point[1], document), convert_values(point[2], document), convert_units(distance, document), angle)

so you then can use

tbdata = [
    # ...
    ['Project Base Point Coordinates to Internal Origin', *report_point(baspt, basptdistance, document)],
    ['Survey Point Coordinates to Internal Origin', *report_point(survpt, surveydistance, document)],
    # ...
]

Another thing that can be done is to transform the convert_values to a function that accepts any number of values to convert and convert all the coordinates at once:

def convert_values(*values, document=doc):
    # ui_units logic unchanged...
    # ...
    return [
        "{:.3g}".format(DB.UnitUtils.ConvertFromInternalUnits(value, ui_units))
        for value in values
    ]

and then

def report_point(point, distance, document):
    angle = calculate_angle(point, INTERNAL_ORIGIN)
    return (*convert_values(*point, document=document), convert_units(distance, document), angle)

you could even parametrize the message and add it to the report_point function, so that you can just use

tbdata = [
    # ...
    report_point("Project Base Point ", baspt, basptdistance, document),
    report_point("Survey Point", survpt, surveydistance, document),
    # ...
]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is crazy good!!
I have this table function to cleanup, I learned a lot of good tricks already.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this unpacking technique is very efficient, I should learn it more and capitalize on it. thank you for sharing.

output.print_md('{} 3D View Bounding Box extends more than {}'.format(BAD_STRG, CRITERIA_STRG))
else:
output.print_md('{} 3D View Bounding Box is located less than {}'.format(GOOD_STRG, CRITERIA_STRG))
test_score = 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't use a global variable, just return 0 or 1 and and do the test_score calculations in the main function

Copy link

@tay0thman tay0thman Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea; I am returning an integer of 0 or 1 and add them in the start test function:


    def startTest(self, doc, output):
        with revit.DryTransaction("Checking Model Extents"):
            check_model_coordinates(doc)
            task_1 = check_model_extents(doc)
            task_2 = check_design_options(doc)
            if task_1 + task_2 < 2:
                output.print_md(
                    ('{} 3D View Bounding Box extends more than {}'.
                     format(BAD_STRG, CRITERIA_STRG)))
                check_linked_elements(doc)
                check_all_elements(doc)
            else:
                output.print_md(
                    ('{} 3D View Bounding Box is located less than {}'.
                     format(GOOD_STRG, CRITERIA_STRG)))

badelements = get_far_elements(document,False, False, True)
counter = 0
limit = 10
if len(badelements) > 0:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if len(badelements) > 0:
if badelements:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done and applied the same to badcads and badrvts

Comment on lines 571 to 577
output.print_md('{} Elements below are located more than {}'.format(BAD_STRG, CRITERIA_STRG))
for x in badelements:
print(output.linkify(x.Id)+ ' ' +
str(x.Name) + ' ' + str(x.Category.Name))
if counter == limit:
break
counter += 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since you don't need the rest of the elements, just keep the first limit elements, no need to keep checking if you reach the limit

Suggested change
output.print_md('{} Elements below are located more than {}'.format(BAD_STRG, CRITERIA_STRG))
for x in badelements:
print(output.linkify(x.Id)+ ' ' +
str(x.Name) + ' ' + str(x.Category.Name))
if counter == limit:
break
counter += 1
badelements = badelements[:limit]
output.print_md('{} Elements below are located more than {}'.format(BAD_STRG, CRITERIA_STRG))
for x in badelements:
print(output.linkify(x.Id)+ ' ' +
str(x.Name) + ' ' + str(x.Category.Name))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

such a beautiful trick

tay0thman and others added 2 commits November 26, 2024 16:13
Cleaned up functions,
Removed redundancy,
streamlined tabulation of coordinates, distances, and angles.
used regions to divide functions.
other formatting items
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Extension Checks Issues related to pyRevit extension checks [subsystem]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants