nulliqhttps://nulliq.dev/Recent content on nulliqHugo -- gohugo.ioen-usPatrick O'NeillSun, 10 Jul 2022 13:08:35 -0500Battleshiphttps://nulliq.dev/posts/battleship/Sat, 12 Mar 2022 08:00:30 -0500https://nulliq.dev/posts/battleship/Let’s overengineer a childhood game.<blockquote>
<p>2022-12-20 Update - There is now an accompanying <a href="https://app.nulliq.dev/battleship">webapp</a>!</p>
</blockquote>
<p>I’ve been playing battleship a lot recently. My friends and I have given up on playing pool and are pretty much exclusively playing this game.</p>
<p>After winning a ton of games by luck, I figured I could do even better using a strategy.</p>
<h1 id="rules">Rules</h1>
<p>Since there are a few different rulesets, here’s a quick overview of the ruleset I was playing with on GamePigeon’s iMessage version of the game.</p>
<ol>
<li>The board is 10x10 squares.</li>
<li>There are 1 len-4, 2 len-3, 3 len-2, and 4 len-1 ships.</li>
<li>You cannot place ships adjacent or diagonal to another ship.</li>
</ol>
<h1 id="naive-strategy">Naive Strategy</h1>
<p>An obvious strategy for Battleship is to shoot where the ships are most likely to be placed.
To calculate this probability, we can enumerate over all the possible ships on the board.</p>
<asciinema-player src="https://nulliq.dev/shippositions.cast" autoplay cols=19 ></asciinema-player>
<p>While we do this, count the number of times a ship occupies each specific square. Then, we can make that array into a heat map that shows the probability of a ship being in that particular square.</p>
<asciinema-player src="https://nulliq.dev/shipcounts.cast" autoplay cols=29 ></asciinema-player>
<p>Since there are 1 len-4, 2 len-3, 3 len-2, and 4 len-1 ships, we will increment each square by the number of ships that could be placed there. That’s why the squares are incrementing by values other than 1.</p>
<p>We end up with a heatmap that looks like this:</p>
<figure class="left" >
<img src="https://nulliq.dev/init_heatmap.png" />
<figcaption class="center" >Dark squares are less likely to have a ship, and pink squares are more likely.</figcaption>
</figure>
<p>We can do this mid-game to get a heatmap specific to that board:</p>
<figure class="left" >
<img src="https://nulliq.dev/basic_heatmap.png" />
<figcaption class="center" >All the dark squares are confirmed hits/misses.</figcaption>
</figure>
<p>We’ll be referring to these values as “probability scores” from here on out. Just remember that “probability scores” are the number of ships that could occupy a square.</p>
<h1 id="information-gain">Information Gain</h1>
<p>Since you cannot place a ship adjacent or diagonal to another ship, all the squares around a hit ship are guaranteed to be a miss. We can exploit this aspect of the game by tracking how many ship positions would be eliminated if a ship occupied that square.</p>
<figure class="left" >
<img src="https://nulliq.dev/sum_board.png" />
<figcaption class="center" >Add up the probabilities surrounding a hit to get an information gain for that square</figcaption>
</figure>
<p>Information gain can be combined with the Naive Strategy in order to calculate both the probability of a hit, along with the value that hit would provide (how many possible positions it would eliminate).</p>
<figure class="left" >
<img src="https://nulliq.dev/sum_board_2.png" />
<figcaption class="center" >This is what we do for a 2x1 ship</figcaption>
</figure>
<p>For each square in each ship position, we add up the surrounding probabilities for that position.</p>
<asciinema-player src="https://nulliq.dev/shippositions.cast" autoplay cols=19 ></asciinema-player>
<asciinema-player src="https://nulliq.dev/infosum.cast" autoplay cols=59 ></asciinema-player>
<p>As a heatmap:</p>
<figure class="left" >
<img src="https://nulliq.dev/infosum_heatmap.png" />
<figcaption class="center" >With this heatmap, the center 4 squares are the best place for your first shot</figcaption>
</figure>
<p>Again, we can do this process mid-game to get a recommendation on where to fire next.</p>
<figure class="left" >
<img src="https://nulliq.dev/infosum_midgame.png" />
<figcaption class="center" >Compared to the naive strategy, this new strategy shows stronger preferences for certain squares</figcaption>
</figure>
<h1 id="sinking-ships">Sinking Ships</h1>
<p>Once we have a hit, what do we do next?</p>
<p>We already know the squares surrounding our hit are eliminated, so let’s pick the next square by only looking at the probability scores for the squares that weren’t already eliminated.</p>
<figure class="left" >
<img src="https://nulliq.dev/single_square_hit.png" />
<figcaption class="center" >Get the sum of these squares to find the most promising second shot.</figcaption>
</figure>
<p>Add them together to get the scores for each direction.</p>
<p>Since the upper pink squares are the brightest, they have the best probability. Let’s fire in the upper direction.</p>
<figure class="left" >
<img src="https://nulliq.dev/double_square_hit.png" />
<figcaption class="center" >That was a hit. Now that we know the ship is vertically oriented, only consider up and down as valid directions.</figcaption>
</figure>
<p>Continue until we discover the whole ship.</p>
<h1 id="optimal-sunk-cost-strategy">Optimal Sunk-cost Strategy</h1>
<figure class="left" >
<img src="https://nulliq.dev/compilicated_board_2.png" />
<figcaption class="center" >The Sinking Ships step says we should fire upwards.</figcaption>
</figure>
<p>We can adjust the previous step to account for the remaining ships. For example, in a situation like the one above, a hit says we should fire upwards. However, if we know there are only len 3 ships left, we can instead fire directly to the right twice for the maximum info gain.</p>
<p>Implementing the relative probability of each ship length occupying the n+1th square, we get the best possible algorithm that I can come up with.</p>
<figure class="left" >
<img src="https://nulliq.dev/compilicated_board_3.png" />
<figcaption class="center" >Firing to the right is the best shot according to the algorithm.</figcaption>
</figure>
<h1 id="closing-thoughts">Closing Thoughts</h1>
<p>This was a fun hobby project. Feel free to try out the code on your own and make pull requests to improve it.</p>
<p>Play around with the webapp version <a href="https://app.nulliq.dev/battleship">here</a>.</p>
<p><a href="https://github.com/Carbocarde/battleship">Github Repo</a></p>
<p>A wish-list item I have is to create a simulator that implements these strategies and measure how well they do against random board layouts. Feel free to tackle this if you have a spare weekend. I’d love to add it to the repo.</p>Polling for Matcheshttps://nulliq.dev/posts/matchy-matchy/Sun, 10 Jul 2022 13:08:35 -0500https://nulliq.dev/posts/matchy-matchy/Can we pair up?<p>I like polls. They’re easy to interact with, and if you have a big groupchat, you can get a lot of responses.</p>
<p>For Valentine’s day this year, I decided to make a script I could run to pair together people who answer polls similarly.
This way people could become friends with people that they are compatible with.</p>
<h1 id="gathering-data">Gathering Data</h1>
<p>When someone answers a poll, let’s assign them a score relative to the other participants.</p>
<p>How we do this is somewhat arbitrary, so let’s give an example poll:</p>
<table>
<thead>
<tr>
<th>Ideal Hangout?</th>
<th>Camping</th>
<th>Fairground</th>
<th>Fancy Dinner</th>
<th>Concert</th>
</tr>
</thead>
<tbody>
<tr>
<td>Camping</td>
<td>5</td>
<td>2</td>
<td>2</td>
<td>4</td>
</tr>
<tr>
<td>Fairground</td>
<td>2</td>
<td>5</td>
<td>3</td>
<td>2</td>
</tr>
<tr>
<td>Fancy Dinner</td>
<td>2</td>
<td>3</td>
<td>5</td>
<td>3</td>
</tr>
<tr>
<td>Concert</td>
<td>4</td>
<td>2</td>
<td>3</td>
<td>5</td>
</tr>
</tbody>
</table>
<p>This is what I call a <code>mapping</code>. It determines the score that two people get when they answer a question. For example, if Person A says <code>Camping</code> and Person B says <code>Fancy Dinner</code>, we’ll give them a score of 2, since those are very different activities.</p>
<p>When asking multiple questions, just add together the scores from each question to get a total score.</p>
<h1 id="data-cleanup">Data Cleanup</h1>
<p>After asking a few questions, we now have a pretty well-defined series of connections.</p>
<p>One problem is if someone didn’t answer all the questions. Their connection scores might look similar to someone who is less compatible.</p>
<p>To fix this, we can first delete anyone who didn’t answer at least a few polls.</p>
<p>Next, we can convert the scores to percentages of the max score possible. That way people aren’t penalized for having answered one less question than their peers.</p>
<h1 id="generating-pairings">Generating Pairings</h1>
<p>So it turns out that pairing people together is the same problem as the <code>maximum matching problem</code><sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. We want to pair people together, but we want everyone to get a good matching. For example, consider this pairing:</p>
<pre tabindex="0"><code>[Jessie] -[12]- [Patrick]
| |
[8] [9]
| |
[Robin] --[3]-- [Cameron]
</code></pre><p>An approach that just chose the highest pairing would pair Jessie & Patrick, at the expense of Robin & Cameron. A better overall pairing would be Jessie & Robin, and Patrick & Cameron.</p>
<p>Thankfully, we don’t need to solve this problem. We can just use the <code>max_weight_matching</code> algorithm in networkx<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>, which avoids the trap and gives the ideal answer overall.</p>
<h1 id="conclusion">Conclusion</h1>
<p>In the future, I would love to poll people who are friends vs people who have stopped hanging out. After running it through some machine learning, the script could learn to assign better pair scores to each user over time. It could also learn if certain questions give better information than others.</p>
<p><a href="https://github.com/Carbocarde/MatchyMatchy">Repo Link</a></p>
<p>The code is set up to accept the copy-pasted poll results from GroupMe.</p>
<p>When you post your polls, make sure you make the responses visible so you can tie each answer to a person :)</p>
<div class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1">
<p><a href="https://dl.acm.org/doi/pdf/10.1145/6462.6502">Efficient Algorithms for Finding Maximum Matching in Graphs</a> <a href="#fnref:1" class="footnote-backref" role="doc-backlink">↩︎</a></p>
</li>
<li id="fn:2">
<p><a href="https://networkx.org/documentation/stable/reference/algorithms/generated/networkx.algorithms.matching.max_weight_matching.html">max_weight_matching implementation</a> <a href="#fnref:2" class="footnote-backref" role="doc-backlink">↩︎</a></p>
</li>
</ol>
</div>Vector Angle: 4 - Benchmarkinghttps://nulliq.dev/posts/vector-angle-4/Sat, 19 Mar 2022 08:00:30 -0500https://nulliq.dev/posts/vector-angle-4/In search of a better dot product<p>This is the final post (#4) of my Vector Angle series. Make sure you check out the <a href="https://nulliq.dev/posts/vector-angle-1">first post</a> for a quick introduction!</p>
<p>So far we’ve covered 3 methods for calculating the angle formed by 2 vectors.</p>
<ol>
<li>Standard (<a href="https://nulliq.dev/posts/vector-angle-1">Law of Cosines</a>)</li>
<li>Visual (<a href="https://nulliq.dev/posts/vector-angle-2">Change of Basis</a>)</li>
<li>Alternative (<a href="https://nulliq.dev/posts/vector-angle-3">Projection</a>)</li>
</ol>
<p>Unfortunately, since we didn’t derive a general formula for the change of basis calculations, I’m going to exclude it from this benchmarking process.</p>
<h1 id="analysis">Analysis</h1>
<p>Before we write any code, let’s analyze the equations themselves.</p>
<p>Law of Cosines (w/ explicit magnitude calculations)
$$ \theta = \arccos {\bigg(\frac {v \cdot u}{{\sqrt{v\cdot v}}{\sqrt{u\cdot u}}}\bigg)} $$</p>
<p>Projection
$$\theta = \arccos{\bigg(\sqrt{\frac{(\frac{u \cdot v}{u \cdot u}u)^2}{v \cdot v}}\bigg)}$$</p>
<p>We can make the projection method better for computers by rearranging some terms:
$$ = \arccos{\bigg(\sqrt{\frac{(\frac{u \cdot v}{u \cdot u})^2(u\cdot u)}{v \cdot v}}\bigg)}$$
$$ = \arccos{\bigg(\sqrt{\frac{\frac{(u \cdot v)^2}{u \cdot u}}{v \cdot v}}\bigg)}$$
$$ = \arccos{\bigg(\sqrt{\frac{(u \cdot v)^2}{(v \cdot v)(u \cdot u)}}\bigg)}$$</p>
<table>
<thead>
<tr>
<th>Ops</th>
<th>Law of Cosines</th>
<th>Projection</th>
</tr>
</thead>
<tbody>
<tr>
<td>Dot product</td>
<td>3</td>
<td>3</td>
</tr>
<tr>
<td>Division</td>
<td>1</td>
<td>1</td>
</tr>
<tr>
<td>Constant * Constant</td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
</tr>
<tr>
<td>Sqrt</td>
<td><strong>2</strong></td>
<td><strong>1</strong></td>
</tr>
<tr>
<td>Arccos</td>
<td>1</td>
<td>1</td>
</tr>
</tbody>
</table>
<p>Now let’s do this table again, but put it in terms of the input vector’s dimensions ($n$).</p>
<ul>
<li>Dot products are $n$ multiplication and $n-1$ additions.</li>
</ul>
<table>
<thead>
<tr>
<th>Ops</th>
<th>Law of Cosines</th>
<th>Projection</th>
</tr>
</thead>
<tbody>
<tr>
<td>Addition</td>
<td>3n - 1</td>
<td>3n - 1</td>
</tr>
<tr>
<td>Multiplication</td>
<td>3n + <strong>1</strong></td>
<td>3n + <strong>2</strong></td>
</tr>
<tr>
<td>Division</td>
<td>1</td>
<td>1</td>
</tr>
<tr>
<td>Sqrt</td>
<td><strong>2</strong></td>
<td><strong>1</strong></td>
</tr>
<tr>
<td>Arccos</td>
<td>1</td>
<td>1</td>
</tr>
</tbody>
</table>
<p>From this table, it looks like it’ll be a close fight. The only difference is between the number of multiplications and sqrt operations.</p>
<h1 id="wait-a-second">Wait a second…</h1>
<p>If we look at the re-arranged Projection method:
$$\theta = \arccos{\bigg(\sqrt{\frac{(u \cdot v)^2}{(v \cdot v) (u \cdot u)}}\bigg)}$$</p>
<p>and compare it to the Law of Cosines method:
$$ \theta = \arccos {\bigg(\frac {v \cdot u}{{\sqrt{v\cdot v}}{\sqrt{u\cdot u}}}\bigg)} $$</p>
<p>There’s something eerily similar about the two equations.</p>
<p>What if we take the Law of Cosines method, square it, then take the square root?
$$ = \arccos {\bigg(\sqrt{\bigg(\frac {v \cdot u}{{\sqrt{v\cdot v}}{\sqrt{u\cdot u}}}\bigg)^2} \text{ } \bigg)} $$
$$ = \arccos {\bigg(\sqrt{\frac {(v \cdot u)^2}{{(\sqrt{v\cdot v}}{\sqrt{u\cdot u})^2}}} \text{ } \bigg)} $$
$$ = \arccos {\bigg(\sqrt{\frac {(v \cdot u)^2}{{(v\cdot v)}{(u\cdot u)}}} \text{ } \bigg)} $$</p>
<p><strong>Woah.</strong></p>
<p>I wasn’t expecting that.</p>
<p>Both independently derived methods are the same.</p>
<p>The only difference is that the Projection method always provides a positive value to $arccos$, which is the reason why we only get acute angles from that method.</p>
<p>I had planned to dive into some of the assembly code and have a bunch of performance charts, but I guess that isn’t needed now.</p>
<h1 id="why-use-the-law-of-cosines-method">Why use the Law of Cosines method?</h1>
<p>Computers don’t use the alternative arrangement provided by the projection method. One of the main reasons<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> for this is that computers provide a fast reciprocal sqrt op that is faster than a normal sqrt op.</p>
<p>That changes the table above:</p>
<table>
<thead>
<tr>
<th>Ops</th>
<th>Law of Cosines</th>
<th>Projection</th>
</tr>
</thead>
<tbody>
<tr>
<td>Addition</td>
<td>3n - 1</td>
<td>3n - 1</td>
</tr>
<tr>
<td>Multiplication</td>
<td>3n + 1<code>-></code>3n + 2</td>
<td>3n + 2</td>
</tr>
<tr>
<td>Division</td>
<td>1<code>-></code>0</td>
<td>1</td>
</tr>
<tr>
<td>Sqrt</td>
<td>2<code>-></code>0</td>
<td>1</td>
</tr>
<tr>
<td>1/Sqrt</td>
<td>0<code>-></code>2</td>
<td>0</td>
</tr>
<tr>
<td>Arccos</td>
<td>1</td>
<td>1</td>
</tr>
</tbody>
</table>
<p>Division and sqrt functions are expensive for computers, so the ability to eliminate all those ops makes a big difference.</p>
<h1 id="closing-thoughts">Closing Thoughts</h1>
<p>Thanks for coming along with me on this ride, it was fun to take the time to solidify my understanding of these linear algebra concepts.</p>
<p>Strangely enough, the projection method performs better than the Law of Cosines method when it is run using CPython. When implemented with a compiled language like C++, the Law of Cosines method has a faster runtime (as expected).</p>
<p>
<figure class="left" >
<img src="https://nulliq.dev/runtime_comp.png" />
<figcaption class="center" >Benchmark of C++ implementation</figcaption>
</figure>
Disclaimer: This code was written before I discovered the equivalence, so it relies on the compiler to simplify the projection method from this form:
$$\theta = \arccos{\bigg(\sqrt{\frac{(\frac{u \cdot v}{u \cdot u}u)^2}{v \cdot v}}\bigg)}$$</p>
<p><a href="https://github.com/Carbocarde/VectorAngle/tree/main/C%2B%2B">Check out the benchmark code</a></p>
<p>Shoutout to <a href="https://github.com/Qwendu">@Qwendu</a> for writing the C++ benchmarking code after I first posted about this on <a href="https://www.reddit.com/r/programming/comments/qs2fjf/in_search_of_a_better_dot_product/">Reddit</a>.</p>
<p>As always, feel free to try out the code on your own and make pull requests to improve it.</p>
<p><a href="https://nulliq.dev/posts/vector-angle-3">Previous -></a></p>
<div class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1">
<p>Alternative reason: branching caused by obtuse angles <a href="#fnref:1" class="footnote-backref" role="doc-backlink">↩︎</a></p>
</li>
</ol>
</div>Vector Angle: Method 3 - Projectionhttps://nulliq.dev/posts/vector-angle-3/Sat, 19 Mar 2022 08:00:30 -0500https://nulliq.dev/posts/vector-angle-3/In search of a better dot product<p>This is post #3 of my Vector Angle series. Make sure you check out the <a href="https://nulliq.dev/posts/vector-angle-1">first post</a> for a quick introduction!</p>
<h1 id="method-3-projection">Method #3: Projection</h1>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/projection.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>Projection is a cool topic. Explained visually, we’re finding the best approximation for the purple vector, if it had to be on the same line as the orange vector. It’s the closest approximation for a vector constrainted to a given subspace.</p>
<p>If you don’t quite get it yet, don’t worry, we’ll build up some intuition before we dive into the math.</p>
<p>Projection is useful because it allows us to use one of its cool properties. Any vector, when projected onto a line, plane, or n-1 dimensional space is <strong>guaranteed</strong><sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> to form a right angle with the line/plane/whatever.</p>
<video width=70% autoplay playsinline loop>
<source src="https://nulliq.dev/proj_abc.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>Once we compute the projection, we will have two pieces of information. Two sides of a triangle with an identical $\theta$, but now with angle $A$ guaranteed to be 90 deg.</p>
<p>That sounds good, but before we go any further, let’s build an intuition for what projection does.</p>
<h1 id="building-intuition">Building Intuition</h1>
<p>When we’re projecting one vector onto another vector, visualize that we’re projecting the vector’s tip (the point) onto another vector’s subspace (the line). A subspace is basically any point that we can reach by scaling a vector by a constant.</p>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/proj_right_angle.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>The shortest line from a point to a subspace (line) forms a perpendicular angle with the subspace.</p>
<p>This is helpful since a projection is exactly that - the projected point has the shortest possible distance from the subspace to the original point.</p>
<p>We can also reason about the process by looking at the unit vectors of the space.</p>
<p>When we project one unit vector onto this subspace, we calculate the length of the projected vector. Remember, the projection is the closest we can get from the tip of the green vector (the point) to the subspace defined by the pink vector (the line).</p>
<video width=30% autoplay playsinline loop>
<source src="https://nulliq.dev/proj_intuition_i.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>Next, we calculate the length for the projection of the other unit vector.</p>
<video width=30% autoplay playsinline loop>
<source src="https://nulliq.dev/proj_intuition_j.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>Now, when projecting a vector onto that subspace, we can just split it into unit vectors, and multiply it by the projection multiples (1.2x and 0.98x).</p>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/proj_intuition.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>A simple way of thinking about this is that we want the projected vector to reside on the pink vector’s line. To do that, we want to multiply the pink vector by a constant so that it is the closest it can get to the purple vector without leaving the pink vector’s subspace.</p>
<p>With the process above, we’re getting the closest we can for each component of the vector and adding those components together to calculate the answer.</p>
<p>(P.S. If you understand what we’re doing there, try combining that with your understanding of <a href="https://www.youtube.com/watch?v=kYB8IZa5AuE">matricies as linear transformations</a>. You can really understand what projection matricies are doing!<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>)</p>
<p>Now let’s use this right angle in our calculations!</p>
<h1 id="practical-application">Practical Application</h1>
<p><em>Let’s do it</em></p>
<p>Given two vectors $a$ and $b$, let’s project vector b onto the subspace defined by vector $a$.</p>
<p>Here’s the formula for the projection of vector $b$ onto vector $a$:</p>
<p>$$ b_{proj} = \frac{a \cdot b}{|a|^2} a = \frac{a \cdot b}{a \cdot a} a $$</p>
<p>Now we have a way to get the three sides that we need, and we can choose which sides to use in our calculation.</p>
<video width=70% autoplay playsinline loop>
<source src="https://nulliq.dev/proj_abc.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>Side A length formula:</p>
<p>$|v|$</p>
<p>Side B length formula:</p>
<p>$|\frac{u \cdot v}{u \cdot u}u|$</p>
<p>Side C length formula:</p>
<p>$|v - \frac{u \cdot v}{u \cdot u}u|$</p>
<p>Now we could technically choose any of these 2 lengths, so let’s try all 3 possibilities and see which one is simplest.</p>
<p>Sin
$\frac{Opp}{Hyp}$</p>
<p>$$\frac{|v - \frac{u \cdot v}{u \cdot u}u|}{|v|}$$</p>
<p>$$\frac{\sqrt{{(v - \frac{u \cdot v}{u \cdot u}u)}^2}}{\sqrt{(v)^2}}$$</p>
<p>$$\sqrt{\frac{{(v - \frac{u \cdot v}{u \cdot u}u)}^2}{v^2}}$$</p>
<p>$$\sqrt{\frac{{v^2 - 2(v\frac{u \cdot v}{u \cdot u}u)} + (\frac{u \cdot v}{u \cdot u}u)^2}{v^2}}$$</p>
<p>Cos
$\frac{Adj}{Hyp}$</p>
<p>$$\frac{|\frac{u \cdot v}{u \cdot u}u|}{|v|}$$</p>
<p>$$\frac{\sqrt{(\frac{u \cdot v}{u \cdot u}u)^2}}{\sqrt{v^2}}$$</p>
<p>$$\sqrt{\frac{(\frac{u \cdot v}{u \cdot u}u)^2}{v^2}}$$</p>
<p>Tan
$\frac{Opp}{Adj}$</p>
<p>$$\frac{|v - \frac{u \cdot v}{u \cdot u}u|}{|\frac{u \cdot v}{u \cdot u}u|}$$</p>
<p>$$\frac{\sqrt{(v - \frac{u \cdot v}{u \cdot u}u)^2}}{\sqrt{(\frac{u \cdot v}{u \cdot u}u)^2}}$$</p>
<p>$$\sqrt{\frac{(v - \frac{u \cdot v}{u \cdot u}u)^2}{(\frac{u \cdot v}{u \cdot u}u)^2}}$$</p>
<p>$$\sqrt{\frac{{v^2 - 2(v\frac{u \cdot v}{u \cdot u}u)} + (\frac{u \cdot v}{u \cdot u}u)^2}{(\frac{u \cdot v}{u \cdot u}u)^2}}$$</p>
<h1 id="simplify">Simplify</h1>
<p>Alright, so now we know that the simplest option is to use Cos because it is the only option that exclusively uses the adjacent and hypotenuse sides. When using projection, we can only calculate the opposite side after we know the hypotenuse and adjacent sides, so it makes sense to use those sides directly.</p>
<p>$$\text{angle} = \arccos{\bigg(\sqrt{\frac{(\frac{u \cdot v}{u \cdot u}u)^2}{v \cdot v}}\bigg)}$$</p>
<p>This is the simplest that we can make the equation - it’s our final answer!</p>
<h1 id="wait-a-second">Wait a second…</h1>
<p>We spotted a problem in our <a href="https://nulliq.dev/posts/vector-angle-2/#what-happens-if-the-vectors-form-an-obtuse-angle">last blog post</a> regarding obtuse angles. That also happens with this method.</p>
<p>Since the projection can face the opposite direction of the initial line, our $\theta$ will always be acute. We can detect and fix this by calculating if $b$ and $b_{proj}$ face the same way. If they aren’t, then the actual angle is $180 - \theta$.</p>
<h1 id="-postsvector-angle-4-up-next"><a href="https://nulliq.dev/posts/vector-angle-4"><-</a> Up Next</h1>
<p>We just derived an alternative way to use the dot product to calculate the angle between two vectors - using projection as a starting point!</p>
<p>We’ve now covered 3 different methods of calculating the angle formed by two vectors:</p>
<ol>
<li>Standard (Law of Cosines)</li>
<li>Visual (Change of Basis)</li>
<li>Alternative (Projection)</li>
</ol>
<p>Up next, we’re going to benchmark these different methods and see which one is fastest on a computer.</p>
<p><a href="https://nulliq.dev/posts/vector-angle-4"><- Read the next post</a> <a href="https://nulliq.dev/posts/vector-angle-2">Previous -></a></p>
<div class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1">
<p>If the vector is already perpendicular with the line/plane/whatever, then the projected vector will have a length of 0 and the angle will be undefined. Additionally, if the vector already resides in the subspace, the difference between the original and projected vector will be zero, and a triangle will not be formed. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">↩︎</a></p>
</li>
<li id="fn:2">
<p>Projection matricies are so cool. You just define what each component (unit vector) maps to in your subspace. Then, when you have a new vector, you just feed each of its components into the mappings, and you get out the projection. Another way of looking at it is that the matrix collapses the output space to only be that subspace, and the mappings ensure that we get as close as possible to the input vector. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">↩︎</a></p>
</li>
</ol>
</div>Vector Angle: Method 2 - Change of Basishttps://nulliq.dev/posts/vector-angle-2/Fri, 11 Feb 2022 08:00:00 -0500https://nulliq.dev/posts/vector-angle-2/In search of a better dot product<p>This is post #2 of my Vector Angle series. Make sure you check out the <a href="https://nulliq.dev/posts/vector-angle-1">first post</a> for a quick introduction!</p>
<h1 id="method-2-change-of-basis">Method #2: Change of Basis</h1>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/changeofbasis.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>Visually, this is my favorite method. The intuition behind this one is a bit clever. Rather than measure the length of a vector, perform a linear transformation to change the space we occupy. This trick allows us to define the space relative to that vector (pink), so that $\hat{i}$ equals the vector. Then, it’s as simple a task as getting the $\hat{i}$ $\hat{j}$ components of the orange vector. Once we have those two components, we can calculate the angle using using the lengths of the components and the guaranteed right angle formed by those components.</p>
<p>Now about this “guaranteed right angle”. So, it turns out this is only true if your basis space vectors ($\hat{i}$, $\hat{j}$) form a right angle. To ensure this is true, the transformation has to be an <em>Affine Transformation</em> (basically, it can only scale, translate, or rotate the space).</p>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/affinevsnon.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>Practically, this just means that when we rotate/scale $\hat{i}$, we also need to rotate/scale $\hat{j}$ by that same amount.</p>
<p><em>Let’s do it</em></p>
<p><strong>2D Vector Space</strong></p>
<p>First, we define a 2x2 matrix that contains the transform we want. Because we are transforming $\hat{i}$ to be the same as the purple vector, we can just put the purple vector’s components in the first column of the matrix.</p>
<p>$$
\begin{bmatrix}
v_\hat{i} & ?\\
v_\hat{j} & ?
\end{bmatrix}
$$</p>
<p>Alright, so we’ve defined $\hat{i}$’s transformation, now we just need to ensure $\hat{j}$ receives an equal amount of rotation/scaling.</p>
<p>Since we know that the two basis vectors will maintain a 90-degree angle, we can define the second basis vector to be the same amount, but with the $\hat{i}$ and $\hat{j}$ flipped, and the $\hat{j}$ coordinate negative.</p>
<p>$$
\begin{bmatrix}
v_\hat{i} & -v_\hat{j}\\
v_\hat{j} & v_\hat{i}
\end{bmatrix}
$$</p>
<p>Great, let’s try applying this matrix to our vectors:
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/notwhatwewant.mp4" type="video/mp4">
Your browser does not support the video tag.
</video></p>
<p>Oh no! This isn’t right. When we defined the transform, were trying to move $\hat{i}$ onto $v$, but we need to do the opposite. Visually, this is what it should be:</p>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/changeofbasisinverse.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>To get the <em>actual</em> matrix, we just need to invert the one we came up with first. Basically, instead of putting $\hat{i}$ <em>onto</em> $v$, put $v$ <em>onto</em> $\hat{i}$.</p>
<p>The formula for inverting a 2x2 matrix is:</p>
<p>$${\begin{bmatrix}
a & b\\
c & d
\end{bmatrix}}^{-1}=\frac {1} {ad-bc}
\begin{bmatrix}
d & -b\\
-c & a
\end{bmatrix}$$</p>
<p>Let’s apply that to the matrix above:</p>
<p>$$
{\begin{bmatrix}
v_\hat{i} & -v_\hat{j}\\
v_\hat{j} & v_\hat{i}
\end{bmatrix}}^{-1} = \frac {1} {(v_\hat{i}*v_\hat{i})-(-v_\hat{j} * v_\hat{j})}
\begin{bmatrix}
v_\hat{i} & v_\hat{j}\\
-v_\hat{j} & v_\hat{i}
\end{bmatrix} =
\frac {1} {v_\hat{i}^2+v_\hat{j}^2}
\begin{bmatrix}
v_\hat{i} & v_\hat{j}\\
-v_\hat{j} & v_\hat{i}
\end{bmatrix}
$$</p>
<p>Great! Now that we have this transformation, we just need to apply it to vector $u$.</p>
<p>$$u \cdot \frac {1} {v_\hat{i}^2+v_\hat{j}^2}
\begin{bmatrix}
v_\hat{i} & v_\hat{j}\\
-v_\hat{j} & v_\hat{i}
\end{bmatrix}$$
$$= \begin{bmatrix}
u_\hat{i} \\
u_\hat{j}
\end{bmatrix}
\cdot \frac {1} {v_\hat{i}^2+v_\hat{j}^2}
\begin{bmatrix}
v_\hat{i} & v_\hat{j}\\
-v_\hat{j} & v_\hat{i}
\end{bmatrix}$$</p>
<p>$$= \frac {1} {v_\hat{i}^2+v_\hat{j}^2} \begin{bmatrix}
u_{i} * v_\hat{i} + u_{j} * -v_\hat{j} \\
u_{i} * v_\hat{j} + u_{j} * v_\hat{i}
\end{bmatrix}$$</p>
<p>$$= \frac {1} {v_\hat{i}^2+v_\hat{j}^2} \begin{bmatrix}
u_{i} * v_\hat{i} - u_{j} * v_\hat{j} \\
u_{i} * v_\hat{j} + u_{j} * v_\hat{i}
\end{bmatrix}$$</p>
<p>Note: In the <a href="#method-2-change-of-basis">first video</a>, the yellow and teal lines each represent a different component of the output.</p>
<p>Now that we have the vector transformed into the basis space, we can use our trig identities to solve it.</p>
<p>For this example, we have the two sides of a right triangle, so we can look back to SOH CAH TOA to see that we have the opposite and adjacent sides.</p>
<p>Therefore:
$$\tan(\theta) = \frac{\text{opp}}{\text{adj}} = \frac {\frac {1} {v_\hat{i}^2+v_\hat{j}^2} (u_{i} * v_\hat{i} - u_{j} * v_\hat{j})} {\frac {1} {v_\hat{i}^2+v_\hat{j}^2} (u_{i} * v_\hat{j} + u_{j} * v_\hat{i})}$$</p>
<p>We can factor out the pesky $\frac {1} {v_\hat{i}^2+v_\hat{j}^2}$:</p>
<p>$$\tan(\theta) = \frac {u_{i} * v_\hat{i} - u_{j} * v_\hat{j}} {u_{i} * v_\hat{j} + u_{j} * v_\hat{i}}$$</p>
<p>$$\theta = \arctan{\bigg(\frac {u_{i} * v_\hat{i} - u_{j} * v_\hat{j}} {u_{i} * v_\hat{j} + u_{j} * v_\hat{i}}\bigg)}$$</p>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/trigidentities.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<h1 id="what-happens-if-the-vectors-form-an-obtuse-angle">What happens if the vectors form an obtuse angle?</h1>
<p>As you can see in the above example, the angle we calculate is always acute.</p>
<p>If we aren’t measuring the correct angle, we can find it by subtracting to get the obtuse angle: $180 - \theta$.</p>
<p>Thankfully, we know that $v$ is equal to $\hat{i}$. So if $u$’s $\hat{i}$ component is negative, we know the two vectors form an obtuse angle.</p>
<p>$$
\theta=\begin{cases}
\theta = \arctan{\bigg(\frac {u_{i} * v_\hat{i} - u_{j} * v_\hat{j}} {u_{i} * v_\hat{j} + u_{j} * v_\hat{i}}\bigg)}, & \text{if $u_{i} * v_\hat{i} - u_{j} * v_\hat{j} > 0$}\\
\theta = 180 - \arctan{\bigg(\frac {u_{i} * v_\hat{i} - u_{j} * v_\hat{j}} {u_{i} * v_\hat{j} + u_{j} * v_\hat{i}}\bigg)}, & \text{if $u_{i} * v_\hat{i} - u_{j} * v_\hat{j} < 0$}\\
\theta = 90, & \text{otherwise}
\end{cases}
$$</p>
<h1 id="3d-vector-space">3D Vector Space</h1>
<p>First, we define a 3x3 matrix that contains the transformation we want. Because we are transforming $\hat{i}$ to become the same value as the purple vector, we can just put the purple vector’s components in the first column of the matrix.</p>
<p>$$
\begin{bmatrix}
v_\hat{i} & ? & ?\\
v_\hat{j} & ? & ?\\
v_\hat{k} & ? & ?
\end{bmatrix}
$$</p>
<p>Sadly, this is as far as we can get without the equation becoming complicated. Since we only have one vector, we now have an entire plane of possible choices for the second vector. It’s only after we arbitrarily choose this second vector that we can choose the third vector.</p>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/3dchangeofbasis.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>In this example, $v$ is the pink vector. The green and orange vector are the two we want to define.</p>
<p>This is a challenging problem, so we might tackle it in a later blog post.</p>
<h1 id="-postsvector-angle-3-up-next"><a href="https://nulliq.dev/posts/vector-angle-3"><-</a> Up Next</h1>
<p>In the next posts, we’ll continue other methods of calculating the angle with some special approaches, along with visualizations of <em>what</em> exactly we’re doing when we apply linear algebra concepts to this problem.</p>
<p><a href="https://nulliq.dev/posts/vector-angle-3"><- Read the next post</a> <a href="https://nulliq.dev/posts/vector-angle-1">Previous -></a></p>Vector Angle: Intro - Dot Producthttps://nulliq.dev/posts/vector-angle-1/Fri, 28 Jan 2022 13:08:23 -0500https://nulliq.dev/posts/vector-angle-1/In search of a better dot product<p>I have an unreasonable love for linear algebra. It might be because of my great professors, or maybe the fact that it’s a subject that I can visualize and make sense of, but either way, there is something super satisfying about learning why something works the way it does.</p>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/angle.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>A few months ago, I was writing a program that involved computing the angle between two vectors. The program was going to do this a lot (I think roughly 150 million times) with different vectors. I made the (poor) decision to use Python for a compute-heavy task. To make matters worse, this was back before I knew the wonders of <a href="https://numba.pydata.org/">Numba</a>.</p>
<p>To start this project, I did what any good programmer does. I googled how people find the angle between two vectors. The industry standard way of finding the angle between two vectors is a formula that relies on the dot product of two vectors.</p>
<p>The formula looks like this:</p>
<p>$$\theta = arccos \bigg( \frac{a \cdot b}{|a| |b|} \bigg)$$</p>
<h1 id="wait-whats-a-dot-product">Wait, what’s a dot product?</h1>
<p>It’s that little dot between a and b, as shown here: $a \cdot b$</p>
<p>A dot product is when you take two vectors, multiply each vector’s components $\langle \hat{i}, \hat{j}, \hat{k} \rangle$ (think x, y, z) together, and get the sum of the products.</p>
<p>So the dot product of
$$
\begin{bmatrix}
2\\
7
\end{bmatrix}
\cdot
\begin{bmatrix}
-3\\
1
\end{bmatrix}$$
is calculated like this:
$$
=(2*-3) + (7 * 1) = -6 + 7 = 1
$$</p>
<h1 id="the-problem">The Problem</h1>
<p>Computing the angle with that formula was <em>really</em> slow. It took nearly 40 minutes to compute, and I had a time limit of 20 minutes.</p>
<h1 id="delving-deeper">Delving Deeper</h1>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/triangle.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>Given two vectors, I want to get the angle between them. Think of this is as a trig problem. Given two sides of a triangle, find the angle that those two sides form with eachother.</p>
<p>Small problem. If you remember from trig, you need at least 3 pieces of information to calculate the missing angle. It could be two sides and an angle, three sides, or one side and two angles. No matter how you do it, you need three pieces of information<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>.</p>
<p>Sadly, we can’t just plug both sides into a trig function and call it a day. We need to perform some operation on the vectors to get an additional piece of information. We could figure out the length of the vector C (from vector A to vector B), or we could determine one of the other angles.</p>
<h1 id="calculating-side-length">Calculating Side Length</h1>
<p>Calculating side length is the same as calculating the magnitude of a vector. For that, we can just use the vectors components and plug them into the Pythagorean theorem.</p>
<p>$$|v|=\sqrt{{v_\hat{i}}^2+{v_\hat{j}}^2+…+{v_\hat{k}}^2} $$</p>
<p>Great! Now we can get the side length of any vector.</p>
<h1 id="method-1-dot-product">Method #1: Dot Product</h1>
<p>Here’s the formula from earlier:
$$\theta = arccos \bigg( \frac{a \cdot b}{|a| |b|} \bigg)$$</p>
<p>We will derive that using the Law of Cosines<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>.</p>
<p>If it’s been a while since you’ve taken geometry, you may have forgotten about the law of cosines. It allows you to calculate an angle using only the side lengths of a triangle.</p>
<p>$$c^2 = a^2 + b^2 − 2ab \cos (\theta)$$</p>
<p>Translating this to in terms of our vectors.</p>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/triangle.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<p>$${|v-u|}^2 = {|v|}^2 + {|u|}^2 − 2{|v|}{|u|} \cos (\theta)$$</p>
<p>Now, let’s expand the expression using the side length formula we defined earlier.</p>
<p>$$\bigg({\sqrt{{(v_\hat{i} - u_\hat{i})}^2+{(v_\hat{j} - u_\hat{j})}^2}\bigg)}^2 = {\bigg(\sqrt{{v_\hat{i}}^2 + {v_\hat{j}}^2}\bigg)}^2 + {\bigg(\sqrt{{u_\hat{i}}^2 + {u_\hat{j}}^2}\bigg)}^2 − 2{|v|}{|u|} \cos (\theta)$$</p>
<p>$${(v_\hat{i} - u_\hat{i})}^2+{(v_\hat{j} - u_\hat{j})}^2 = {{v_\hat{i}}^2 + {v_\hat{j}}^2} + {u_\hat{i}}^2 + {u_\hat{j}}^2 − 2{|v|}{|u|} \cos (\theta)$$</p>
<p>$${v_\hat{i}}^2 + -2{(v_\hat{i}*u_\hat{i})} + {u_\hat{i}}^2 + {v_\hat{j}}^2 + -2{(v_\hat{j}*u_\hat{j})} + {u_\hat{j}}^2 = {{v_\hat{i}}^2 + {v_\hat{j}}^2} + {u_\hat{i}}^2 + {u_\hat{j}}^2 − 2{|v|}{|u|} \cos (\theta)$$</p>
<p>Remove the duplicate ${u_{i}}^2$ and ${v_{i}}^2$ terms from each side:</p>
<p>$$-2{(v_\hat{i}*u_\hat{i})} -2{(v_\hat{j}*u_\hat{j})} = − 2{|v|}{|u|} \cos (\theta)$$</p>
<p>$${(v_\hat{i}*u_\hat{i})} + {(v_\hat{j}*u_\hat{j})} = {|v|}{|u|} \cos (\theta)$$</p>
<p>$$ v \cdot u = {|v|}{|u|} \cos (\theta)$$</p>
<p>$$ \frac {v \cdot u}{{|v|}{|u|}} = \cos (\theta)$$</p>
<p>$$ \theta = \arccos {\bigg(\frac {v \cdot u}{{|v|}{|u|}}\bigg)} $$</p>
<p>Great! Now we know how the angle relates to the dot product of two vectors.</p>
<p>Also note that if $|u|=|v|=1$, then $|v||u|=1$ and we can reduce the equation to:</p>
<p>$$ \theta = \arccos {({v \cdot u})} $$</p>
<p>This means we could also normalize the vectors (reduce the length to 1) and calculate the angle between those normalized vectors.</p>
<video width=100% autoplay playsinline loop>
<source src="https://nulliq.dev/normalize.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<h1 id="-postsvector-angle-2-up-next"><a href="https://nulliq.dev/posts/vector-angle-2"><-</a> Up Next</h1>
<p>This is the first post in a series. In the next posts, we’ll be exploring other methods of calculating the angle with some special approaches, along with visualizations of <em>what</em> exactly we’re doing when we apply linear algebra concepts to this problem.</p>
<p><a href="https://nulliq.dev/posts/vector-angle-2"><- Read the next post</a></p>
<div class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1">
<p>One of those pieces of information needs to be a side length. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">↩︎</a></p>
</li>
<li id="fn:2">
<p>Here’s a good <a href="https://www.youtube.com/watch?v=QFMqZkwLTpU">proof of the law of cosines</a> <a href="#fnref:2" class="footnote-backref" role="doc-backlink">↩︎</a></p>
</li>
</ol>
</div>Hey there!https://nulliq.dev/about/Sat, 22 Jan 2022 00:00:00 +0000https://nulliq.dev/about/I’m Patrick, a fourth year CS major that goes to Georgia Tech. I’ll be graduating in December 2022.
github linkedin This is my site, where I post from time to time (not often).
While you’re here, why don’t you check out my posts?<p>I’m Patrick, a fourth year CS major that goes to Georgia Tech.
I’ll be graduating in December 2022.</p>
<ul>
<li><a href="https://github.com/Carbocarde">github</a></li>
<li><a href="https://www.linkedin.com/in/Patrick-One">linkedin</a></li>
</ul>
<p>This is my site, where I post from time to time (not often).</p>
<p>While you’re here, why don’t you check out my <a href="https://nulliq.dev/">posts</a>?</p>