Skip to content

Commit

Permalink
Merge pull request #31 from shiftbug/patch-1
Browse files Browse the repository at this point in the history
Revised stochastic-software.md
  • Loading branch information
jxnl authored Jan 23, 2024
2 parents bfc2ef6 + 9f7a220 commit 893c528
Showing 1 changed file with 1 addition and 4 deletions.
5 changes: 1 addition & 4 deletions docs/writing/posts/stochastic-software.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,6 @@ Consider this example: we, a group of data scientists, are analyzing the various
| Scheduling Questions | 90 | 83% |
| Internet Searches | 20 | 9% |

Consider this example: we, a group of data scientists, are analyzing the various types of queries received by a retrieval application. We've classified the queries using a classification model, and we've aggregated data to determine the volume and quality of each query type.

Here are some examples of recommendations that we can make based on this data:

1. Our performance in internet searches is clearly underwhelming, but the count is quite low.
Expand All @@ -101,6 +99,7 @@ That being said, it's also important to trust your judgment. Even if you're goin
Many people transitioning from classical software engineering to machine learning are often surprised by the empirical nature of the results we obtain. Instead of executing discrete unit tests, we sample from the distribution of potential inputs and build a internal model of how this system operates.

I hope that this article has helped you understand the importance of focusing on outcomes, metrics, and experiments instead of trying to think our way through edge cases and long tails. Additionally, I encourage you to develop the habit of making decisions and eliminating other possibilities. Lastly, I hope you will cultivate the practice of documenting your results and sharing them with your team, fostering a collective learning experience.

As you're building these probabilistic systems, ask yourself:

1. Could an experiment just answer my questions?
Expand All @@ -109,8 +108,6 @@ As you're building these probabilistic systems, ask yourself:
4. Under what conditions will I reevaluate if results are not positive?
5. Can I use the results to update my mental model and plan future work?

I hope that this article has helped you understand the importance of focusing on outcomes, metrics, and experiments instead of trying to think our way through edge cases and long tails. Additionally, I encourage you to develop the habit of making decisions and eliminating other possibilities. Lastly, I hope you will cultivate the practice of documenting your results and sharing them with your team, fostering a collective learning experience.

## One more thing

This is a great point that a friend of mine called out. Set due dates for your experimentation. And if you cannot get a result by the due date, that is the result. **Write that down, explain why it takes longer than we expected, and move on.** For now, that is the negative result.

0 comments on commit 893c528

Please sign in to comment.