tag:blogger.com,1999:blog-5794422705482173402024-03-17T18:31:50.241-07:00arcs and tangentsarchttp://www.blogger.com/profile/03694523494127499522noreply@blogger.comBlogger34125tag:blogger.com,1999:blog-579442270548217340.post-43559562307732363942024-03-10T18:28:00.000-07:002024-03-11T20:49:50.296-07:00Digital Transformation, Really?<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxTJQBjeXt90eHjHN8vROKiGWMIcTyfXPuo1BqzDm4dExKFOnS91vkhEIEUSZ82QETTkIE7TytPzDXUoBkuRcnvDOqKzK79sB5AZhODPCK8RppN48eKBlbUdmspdGvTnrhFdz-NhvQp5dylIiINYiXRHh4vh8RBDjAyP4m0UaMPXhjLoewAiW26WFEdHVs/s1024/DALL%C2%B7E%202024-03-10%2020.24.40%20-%20A%20tranquil%20scene%20from%20medieval%20times%20capturing%20monks%20meticulously%20writing%20manuscripts%20in%20a%20dimly%20lit%20monastery%20library.%20The%20room%20is%20filled%20with%20large%20.webp" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="1024" data-original-width="1024" height="262" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxTJQBjeXt90eHjHN8vROKiGWMIcTyfXPuo1BqzDm4dExKFOnS91vkhEIEUSZ82QETTkIE7TytPzDXUoBkuRcnvDOqKzK79sB5AZhODPCK8RppN48eKBlbUdmspdGvTnrhFdz-NhvQp5dylIiINYiXRHh4vh8RBDjAyP4m0UaMPXhjLoewAiW26WFEdHVs/w262-h262/DALL%C2%B7E%202024-03-10%2020.24.40%20-%20A%20tranquil%20scene%20from%20medieval%20times%20capturing%20monks%20meticulously%20writing%20manuscripts%20in%20a%20dimly%20lit%20monastery%20library.%20The%20room%20is%20filled%20with%20large%20.webp" width="262" /></a></div>Judging by the solicitations I get from various vendors, "Digital Transformation" seems to be a big buzz word in the business circles. I find this strange. When the Internet and Web first took off, 20 to 25 years ago, I could see companies needing a "digital transformation". But now? Maybe there's some fringe businesses that still rely on paper, or very large legacy use cases but for any company that is less than 20 years old, what do they mean by trying to sell me digital transformation services? And what larger company has not been working on doing things "digitally" for the last 20+ years? Why is this term still being used? Is it just the consulting industry's ploy to sell us services?<p></p>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-23909721301337270202024-03-07T18:26:00.000-08:002024-03-07T18:26:25.794-08:00The Startup Equation<p>This is what I call the startup equation for staffing. If time is "T" and work is "W", then:</p><p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieAS8ho5cjDL25X5KS3Vwm2qB8vyPs91raz61PyxAgGF5gN73_6N6x5w6eZ7XQyodpa3l5DZjbiIzyrMsRueWvAQXLbhkP-aUT953SOsrKXXkqjkuUhIGt25yYLOi-4A3Q9bdFj4aJivs9hqGYV0I-Yeli6nH1G1AG1M-Wq3h_9Vzf4z9N5DJNdnJX_B5q/s476/CodeCogsEqn.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="23" data-original-width="476" height="19" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieAS8ho5cjDL25X5KS3Vwm2qB8vyPs91raz61PyxAgGF5gN73_6N6x5w6eZ7XQyodpa3l5DZjbiIzyrMsRueWvAQXLbhkP-aUT953SOsrKXXkqjkuUhIGt25yYLOi-4A3Q9bdFj4aJivs9hqGYV0I-Yeli6nH1G1AG1M-Wq3h_9Vzf4z9N5DJNdnJX_B5q/w400-h19/CodeCogsEqn.png" width="400" /></a></div><p><br /></p>At no point in time at any of the startups I worked at did we have enough people to do all the work we needed to be doing. At first I complained, then I understood and now I accept it. If you are at a startup and feel you need more people, understand that it is no one's fault and there is nothing to be done about it: it is just the way the math works.<p></p>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-30715385359212326672021-02-05T17:07:00.000-08:002021-02-05T17:07:33.606-08:00Albers Projection for Maps<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVFJ64Zu3j0yNjHv8s0J7nMNKrO_qsyBRK_-bSS1F0eGeC0CAYEJBxMoQGW2Sby1twb31HUAex2u6uVmnR_JhsNSyhaLz2MrraYfNgn_62ViWelAUQ6l5wAMHnkUqnBhQTdXZWtTv3YbGb/s1397/Screenshot+from+2021-02-05+18-48-06.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="763" data-original-width="1397" height="159" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVFJ64Zu3j0yNjHv8s0J7nMNKrO_qsyBRK_-bSS1F0eGeC0CAYEJBxMoQGW2Sby1twb31HUAex2u6uVmnR_JhsNSyhaLz2MrraYfNgn_62ViWelAUQ6l5wAMHnkUqnBhQTdXZWtTv3YbGb/w292-h159/Screenshot+from+2021-02-05+18-48-06.png" width="292" /></a></div> I wrote this Python code a couple months ago when I needed a simple stand-alone way to plot cities on an SVG representation of a map. There were no good, standalone Python references I could find. Even when I found something in other languages, the readability of it was somewhat lacking. <p></p><p>It seems the more math you know, the fewer characters you put in your variable names. Sigh. Are they using meters or miles? Are they using radians or degrees? You have no idea from looking at the code, so you *have* to understand the math to be able to decipher it.<br /></p><p>Developing this code required some investment of time in not just the math, but the geographic uses of the <a href="Albers projection" target="_blank">Albers projection</a>. I figured it might be generally useful, so have made the code available in GitHub here:</p><p style="margin-left: 40px; text-align: left;"><a href="https://github.com/cassandra/geo_maps" target="_blank">https://github.com/cassandra/geo_maps</a></p><p>It includes an example that deals with some tricky issues that can come up if you want to plot points against a map of the U.S. </p><p>When you see those U.S. maps where they have relocated Alaska and Hawaii to be in the lower left of the map, it is obvious they are not in the right location and are of different scale than the contiguous 48 states. However, did you know that they are using completely different projection parameters *and* have rotated them? I learned this the hard way.<br /></p><p><br /></p>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-7330492779847067842021-02-04T20:47:00.000-08:002021-02-04T20:47:11.821-08:00Starting Over<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHi2pVTLpc_MBrmqwvYhCOx14JeeBKwUhy-pbfYnRp0ANtXtMUeRr6ut-Hq-lDlZ9r6Yyz7E41GSGpWI3JMLxZpegKGX5IdPK9m_t-EbCE-AO4dCGK8NQGjLt2Yqd3u3kdPgoksgTBXtd4/s745/refresh.png" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="745" data-original-width="676" height="217" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHi2pVTLpc_MBrmqwvYhCOx14JeeBKwUhy-pbfYnRp0ANtXtMUeRr6ut-Hq-lDlZ9r6Yyz7E41GSGpWI3JMLxZpegKGX5IdPK9m_t-EbCE-AO4dCGK8NQGjLt2Yqd3u3kdPgoksgTBXtd4/w196-h217/refresh.png" width="196" /></a>I read a thought provoking article:<p></p><p style="margin-left: 40px; text-align: left;"><a href="https://hiringengineersbook.com/post/autonomy/" target="_blank">How to hire senior developers: Give them more autonomy</a></p><p>It contains the provocatively titled section "Your Code is Worthless", and it makes a good case for this being true. The context here is not that the running code is worthless to the business, but that the code itself is of no value to anyone else. i.e., Worrying about a competitor stealing your code base is pointless.<br /></p><p>A basis of the argument is that the software engineering process is not about creating an artifact (the code base), but about building a "theory". Another way to express this idea that I have seen is: the software development process is a group exercise in the organization of information. The code is nothing more that the way to capture the outcome of that collaborative process.</p><p>To reach the conclusion that the resulting code is worthless, the article borrows from work by Peter Naur (of BNF fame) which says it is impossible for software, or its documentation, to encode all the information that was collectively organized and which is necessary to efficiently maintain it. </p><p>There are a lot of good supporting details in that article, so it is definitely worth reading. Many of the points are aligned with my experiences. The one conclusion that got me thinking is the premise that it is cheaper and faster to start from scratch than to try to adopt an unfamiliar code base. I do agree that this is true for a competitor getting a hold of other company's code. But could this apply to the company that owns the code?</p><p>What if a company had 100% turn-over of their developers, is starting over the best choice for the company? <br /></p><p>Engineers usually tend to think a rebuild is the right answer in this case, but
the business side never does. Is the business perspective wrong?<br /></p><p>I
have been in a situations where there was 100% turnover in a
complicated code base. I pushed to keep some of the previous developers
on contract because I knew how crucial it was to leverage the
information in their brains. Even if a rewrite was the right answer,
this was not going to happen instantaneously and the business needed to
keep operating. </p>So maybe it is this continuity aspect that changes the equation. If you have no choice but to
figure out the code base, then a rewrite is only an added cost, not a
replacement cost.archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-90419465577021621202021-02-01T14:53:00.001-08:002021-02-01T14:56:53.381-08:00Can We Please Keep it Simple?<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwaYtxeNWFu6l5x-DeBncPJYVDdpbm6qq-Z_9SWqn6b4li9QoNamWRSwmuq_JZryBrUrhekY96rGjJxxQsQ1vOkL_7jfQS78RqyTaSe0wsrA0wG5L38gBEiKmI2oVr2rYeDMGF16mS6Bgj/s596/simple.jpeg" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="271" data-original-width="596" height="100" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwaYtxeNWFu6l5x-DeBncPJYVDdpbm6qq-Z_9SWqn6b4li9QoNamWRSwmuq_JZryBrUrhekY96rGjJxxQsQ1vOkL_7jfQS78RqyTaSe0wsrA0wG5L38gBEiKmI2oVr2rYeDMGF16mS6Bgj/w220-h100/simple.jpeg" width="220" /></a></div><br />There is a famous quote about software that I have heard attributed to Brian Kernighan:<p></p><p style="margin-left: 40px; text-align: left;"><i>"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."</i></p><p style="text-align: left;">I do not think you would find a software developer that disagrees with that statement, yet the field is overrun by a culture that seems to value succinctness more than readability.</p><p style="text-align: left;">When I began to write a lot of Perl, I wrote it like I wrote C programs and caught the ridicule of the Perl wizards who told me it was "better" to rewrite it as something that would wind up looking like this: <br /></p><pre style="margin-left: 40px; text-align: left;">@P=split//,".URRUU\c8R";@d=split//,"\nrekcah xinU / lreP&&
close$_}%p;wait until$?;map{/^r/&&<$_>}%p;$_=$d[$q];</pre><p style="text-align: left;">I continued to write the code simply with the fewest "Perl-isms" I could. Our hiring budget would only allowed us to hire mere mortals to help maintain it. <br /></p><p style="text-align: left;">Ten year later, the pattern repeats itself. When I write Python, people point out all the extra syntax I do not need, even though it is my deliberate attempt to make things more explicit and readable. Optimizing character and line counts seems to be a reflex for many programmers. <br /></p><p style="text-align: left;">I have grown to abhor this sentence:</p><p style="margin-left: 40px; text-align: left;"><i>"Look what I can do in just one line of code."</i></p>This
is usually pitched to me in the context of someone trying to convince
me of the wonders of the latest, greatest programming language. <p style="text-align: left;">Of all the languages I have learned, there have been only a handful of language advances I have encountered where truly "better" syntax was introduced in terms of readability. Some of them include:</p><ul style="text-align: left;"><li>try-catch exception handling</li><li>"finally" blocks</li><li>"else" for for loops</li><li>list comprehensions (but only if used judiciously)</li><li>string interpolations</li></ul><p>In general, if the syntactic feature is specific to a language, I will avoid using it. I shift around writing code in many languages and the more I can do to minimize the context shift, the better.<br /></p><p style="text-align: left;">With software, readability is 99% of the problem to be solved. As Martin Fowler has said:</p><p style="margin-left: 40px; text-align: left;"><i>"Any fool can write code that a computer can understand. Good programmers write code that humans can understand."</i></p><p style="text-align: left;">I was inspired to share these thoughts after reading this article that also makes the case for keeping it simple:</p><p style="margin-left: 40px; text-align: left;"> <a href="https://lemire.me/blog/2017/12/06/simplistic-programming-is-underrated/" target="_blank">Simplistic programming is underrated</a></p><p>I have also recently read this related article:</p><p style="margin-left: 40px; text-align: left;"><a href="https://blog.feenk.com/developers-spend-most-of-their-time-figuri-7aj1ocjhe765vvlln8qqbuhto/" target="_blank">Developers spend most of their time figuring the system out</a><br /></p><p>This is definitely in line with my experiences and helps to emphasize how important code readability truly is.<br /></p><p><br /></p>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-608373328231791232021-01-10T13:45:00.001-08:002021-02-01T14:01:57.937-08:00Collective Fictions of Software Methodologies<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2TEW1ywx0VFni0W18xsNfykDMFv54nvyCPHaiFYXykbD_G88juW4QbeDpQvm_1P6eI2nX2gWV8Z-msuRTVtJlwvWd60DGWviey0wl71v5qOsI6_JxAwYSr9JGGRs_MrkrkSdCHf4Gi4L9/s320/320px-Waterfall_model.svg.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="240" data-original-width="320" height="157" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2TEW1ywx0VFni0W18xsNfykDMFv54nvyCPHaiFYXykbD_G88juW4QbeDpQvm_1P6eI2nX2gWV8Z-msuRTVtJlwvWd60DGWviey0wl71v5qOsI6_JxAwYSr9JGGRs_MrkrkSdCHf4Gi4L9/w209-h157/320px-Waterfall_model.svg.png" width="209" /></a></div><br />Software methodologies are a collective fiction. Necessary, but a fiction nonetheless. The author of this article is where I first saw this concept:<p></p><p style="margin-left: 40px; text-align: left;"><a href="https://zwischenzugs.com/2017/10/15/my-20-year-experience-of-software-development-methodologies/" target="_blank">My 20-Year Experience of Software Development Methodologies</a><br /></p><p>My experiences and conclusions have been similar. A software team needs something to organize them, but exactly what is used does not matter that much. Really, all you need is:<br /></p><ol style="text-align: left;"><li>A system to organize the work</li><li>A system to communicate. </li></ol><p>All the software methodologies provide these elements. The bottom line is that any methodology can work with good people and every methodology will fail with bad people. </p><p>As one of the cartoons in that article alludes to: the goal of a methodology is predominantly about avoiding chaos.</p><p> </p>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-61690259158549078652020-12-20T08:52:00.062-08:002021-02-01T12:48:14.324-08:00Microservices - Why?<p> </p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmxhMfhmYHGpwl2WNrnKG4tIWLvbVIyCNr9EpAVuLRREtXMCwkKeFe5O0Zb2naJVX3u3xNqbaeAH4ybFwfJecZloVG9EV9j7FuwnJit1Kb7KS9vQn4aR6FsOmJ33TgxizOv_ZQo-Jymfp8/s200/shiny.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="200" data-original-width="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmxhMfhmYHGpwl2WNrnKG4tIWLvbVIyCNr9EpAVuLRREtXMCwkKeFe5O0Zb2naJVX3u3xNqbaeAH4ybFwfJecZloVG9EV9j7FuwnJit1Kb7KS9vQn4aR6FsOmJ33TgxizOv_ZQo-Jymfp8/s0/shiny.png" /></a></div><br />I read an article with the basic theme: <br /><p></p><p style="margin-left: 40px; text-align: left;">“Microservices was a good idea taken too far and applied too bluntly."</p><p style="margin-left: 40px; text-align: left;"><i>Link: <a href="https://vlfig.me/posts/microservices/">Microservices — architecture nihilism in minimalism's clothes</a></i></p><p>I agree with a lot of what is in there as I have seen the complications this design pattern leads to and often wondered if it was worth it. </p><p>This general behavior matches a familiar one that happens time and again. A similar things happened (is still happening?) with NoSQL and you could say the same thing:</p><p style="margin-left: 40px; text-align: left;"> “NoSQL was a good idea taken too far and applied too bluntly."</p><p style="text-align: left;">Ditto for the semi-recent trend of "Single Page Webapps". XML was another one. The list is broad, deep and goes further back than my time in the field.<br /></p><p>Why is there the constant trend in software development of chasing the
latest idea and overusing it until it collapses on itself? Haven't we seen this pattern enough to not repeat our mistakes? Ultimately, this is an
immature behavior, but what is the source of that immaturity? Is it
simply because the field only goes back a few decades? Is it that the
field is dominated by a younger and less experienced group of
people?<br /></p><p>Software engineers are initially drawn to the discovery aspect of technology, so they naturally gravitate toward the "new". Human nature leans toward doing the familiar, often ignoring the "why". There is also the time required to learn a new technology that is a driver for getting the most out of that investment. Maybe these combine so that once the new thing is learned, it tends to propagate that new pattern and/or technology without asking "why".</p><p>Maybe it is an education problem. Are we giving students these lessons
and warnings about them in their Software Engineering classes? Wouldn't it be helpful if every student coming out of school knew about this danger and could recognize it as quickly as they can regurgitate the big-oh complexity of a bubble sort? <br /></p><p>There's also a class culture in software where the "coolness" factor is related to the "newness" factor. Many engineers look down upon the use of older technologies, ridiculing their use and sometimes shaming people into using new tech. Who wants to be coding in PHP and be socially outcast from all the cool kids? This behavior is especially troubling because it promotes the idea that the technology is more important than the problem it is meant to solve. <br /></p><p>As an engineering leader, it is important to combat these less-than-rational reasons for adopting a technology. I think I am often viewed as a curmudgeon about adopting new technology, but asking "why" is the responsible thing to do. <br /></p><p> </p>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-45192488950990952512020-11-08T12:48:00.006-08:002021-02-03T14:52:41.286-08:00Code Reuse is Overrated<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAtie2_Bl-mslIYcRUBDfqyZgu0gV9-vCUv4TF_w_su0eAr5CycGUgD_RhVdLSy291X-BZk2eCCK2ovIo2myK0wlQHVE09WWM2WFYWP40LecOdztynM2IIrWufw2JbEVCKRzn-AXLISxC3/s1280/arrows-3368751_1280.png" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="1280" data-original-width="1277" height="161" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAtie2_Bl-mslIYcRUBDfqyZgu0gV9-vCUv4TF_w_su0eAr5CycGUgD_RhVdLSy291X-BZk2eCCK2ovIo2myK0wlQHVE09WWM2WFYWP40LecOdztynM2IIrWufw2JbEVCKRzn-AXLISxC3/w160-h161/arrows-3368751_1280.png" width="160" /></a></div><br />As an engineer, I have often been conflicted about the interplay of code reuse and dependency management. It is innate behavior for me to not want to repeat myself and it is through hard and painful experiences that I have learned to be hyper-critical of all new dependencies.<p></p><p>Why re-invent the wheel? Because the existing wheel comes with lots of strings attached. This article does a good job of explaining the pitfalls in depth:</p><p style="margin-left: 40px; text-align: left;"><a href="http://yosefk.com/blog/redundancy-vs-dependencies-which-is-worse.html" target="_blank">Redundancy vs dependencies: which is worse?</a><br /></p><p>If it is my wheel and it breaks, I know how to and am able to fix it quickly and cheaply (measured in time). If I depend on someone else's wheel, now I have lots of problems. Will they fix it and when? Will their new version introduce new things I am not expecting? Maybe they too have their own dependencies. That's to name just a few.</p><p>I was on a team where a large third-party Java library was added to the project for the sole purposes of using the "isBlank()" function. That was not a good trade-off of re-use and dependencies. I hear the Javascript/npm world has this same sort of problem in spades.<br /></p><p>Too often the Don't Repeat Yourself (DRY) mantra is used as a gospel, devoid of the dependency cost. I've seen development cultures where adding dependencies is done often and effortlessly: it does not even register that there should be a decision process around this. As a group, we need to be more thoughtful about the trade-offs we are making when introducing a dependency and to assign it the proper cost.</p><p>Here is another good article related to this theme:</p><p style="margin-left: 40px; text-align: left;"><a href="https://copyconstruct.medium.com/small-functions-considered-harmful-91035d316c29" target="_blank">Small Functions considered Harmful</a><br /></p>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-13270751528201719082020-11-05T08:02:00.002-08:002021-02-05T09:35:01.848-08:00Fallacies of Distributed Computing<div class="separator"><p style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"> <img height="188" src="https://upload.wikimedia.org/wikipedia/commons/thumb/1/12/Cloud_computing_icon.svg/200px-Cloud_computing_icon.svg.png" style="-webkit-user-select: none; background-color: #e6e6e6; margin: auto; transition: background-color 300ms ease 0s; user-select: none;" width="269" /></p></div><p>I came across this Wikipedia page:</p><p style="margin-left: 40px; text-align: left;"><a href="https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing" target="_blank">Fallacies of Distributed Computing</a></p><p>It lists the fallacies as:</p><ul style="text-align: left;"><li>The network is reliable;<br /></li><li>Latency is zero;<br /></li><li>Bandwidth is infinite;<br /></li><li>The network is secure;<br /></li><li>Topology doesn't change;<br /></li><li>There is one administrator;<br /></li><li>Transport cost is zero;<br /></li><li>The network is homogeneous. <br /></li><li>We all trust each other.<br /></li></ul><p>These are all good things to keep in mind while you design a distributed system, but I think the use of the word "fallacy" is a bit overstated. I've seen a lot of designs (and existing systems) where some of those items have been neglected, but the reason is not because the author had "mistaken beliefs". </p><p>Even for someone new to distributed systems, if you asked them "Is the network reliable?", they will rightly know that it is not. If their first designs do not properly account for this, it is not because they had mistaken beliefs, but more due to their inexperience or oversight.</p><p>The same is true for the remaining items: if you asked someone the specific question, you will likely get the right answer, though their designs may still be lax in that area. </p><p>If you forgot to pay your electric bill, I would not conclude that you have the false belief that electricity is free.<br /></p><p><br /></p>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-72810836035109059352020-10-25T14:04:00.002-07:002021-01-31T23:11:04.196-08:00Scalable Search Engines<p>I have now worked at three different companies where we needed to provide a search feature that could scale into the millions of documents. Two of these involved consumer products and the other consisted of documents for legal review. I saw a very interesting contrast in the requirements between these two domains that I thought might be worth sharing.<br /></p><h2 style="text-align: left;">The Use Cases <br /></h2><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRo_gVvHf-jSY_28OxO_qSgM2Gt_cFpUdUlxqqlV7w8HAzZFYemlbcocW533c7zAuw9m6da3JKhvcBU_GHPG4jGRKOSEZPoQVu945-8tCmYRM2cTi1_M1-i5iGIuMeziqThSO5Bid3GObw/s1920/search-magnifying-glass.png" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="1519" data-original-width="1920" height="132" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRo_gVvHf-jSY_28OxO_qSgM2Gt_cFpUdUlxqqlV7w8HAzZFYemlbcocW533c7zAuw9m6da3JKhvcBU_GHPG4jGRKOSEZPoQVu945-8tCmYRM2cTi1_M1-i5iGIuMeziqThSO5Bid3GObw/w166-h132/search-magnifying-glass.png" width="166" /></a></div><p></p><p>In the consumer product space, we had millions of users a day searching across hundreds of millions of documents, while in the legal technology space, we only had a few thousand users a day who were only searching across tens of millions of products. </p><p>At this level, it would seem like the legal domain is a much easier system design problem to solve. However, there are a couple of key difference between the two use cases which can make the legal search problem much, much harder.</p><h2 style="text-align: left;">Data Shapes<br /></h2><p>Consumer products result in fairly small documents: there is a title, some attributes and maybe a description that need to be indexed. For legal review, this could involve any type of document: if it is on someone hard drive or in their cloud accounts, it may be subject to legal review. Out in the wild, there are Excel data files in excess of 120MB and PDF files with tens of thousands of pages. They all need to be in the search index. (Processing files that exhibit these extremes is a nightmare in itself, but here we'll stick with the search-related problems.)</p><p>Besides the potential for these large documents, the big issues that makes the system design hard is the variance of the sizes. There are plenty of small documents to go with these these larger documents in a legal review. In contrast, consumer products are relatively uniform in the size of what needs to be indexed. <br /></p><h2 style="text-align: left;">Query Shapes</h2><p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhppfrMYRvYXAJP5Frh2cu10IG6Le8lUEMy5yVqjlZKidB9vOakDF7cToVZCjtkiwB5AV703aCT3ikxBh6eLKBGqXdqvvLerqWb8db-vRFk_2MTVK6R8vQTUXycXuQcGB9Kvso78qf3VZCV/s297/search-syntax.png" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="297" data-original-width="170" height="243" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhppfrMYRvYXAJP5Frh2cu10IG6Le8lUEMy5yVqjlZKidB9vOakDF7cToVZCjtkiwB5AV703aCT3ikxBh6eLKBGqXdqvvLerqWb8db-vRFk_2MTVK6R8vQTUXycXuQcGB9Kvso78qf3VZCV/w139-h243/search-syntax.png" width="139" /></a></div><p></p><p>With consumer products, the search expressions that are typical will consist of a handful of words at most (on average, about 2.3 words). Users mostly express simple concepts like "Sony 55 in. TV". In the legal space, queries are not only much longer, but a considerable amount of time is spent crafting the searches to hone in on specific concepts. Search expression are serious business for lawyers. Competing parties will negotiate search terms and argue about them before judges. It is not uncommon for a single search to have hundreds of terms.</p><p>The size of the search expression is only a small part of the story though. The legal domain requires all the special search operators might can think of: wildcard searches, proximity searches, range searches, etc. Not only do they require these, these are used extensively. </p><p>Consider search for a specific person of interest in a legal case whose full legal name is "Larry Thomas Johnson". How do you search for documents that mention this name so as to not miss any? "Larry Johnson", "L. Johnson", "L. T. Johnson", "Larry T. Johnson" are some of the more obvious variations that might be out there. In the legal world, they will do something like this: "L? \2 Johnson", where "?" is a wildcard matching any suffix and "\2" is a proximity search looking for occurrences within 2 words.</p><p>Requiring these sophisticated queries makes perfect sense in the legal context, but they are also the types of queries that stress a search engine (in both CPU and memory resources). The contrast of these two domains is between a high volume of simple searches and a lower volume of more complex searches. </p><p>As with the data shapes, in the legal space the query shapes tend to have a very high variance. There are plenty of simple queries to mix in with the more sophisticated ones. A handful of sophisticated queries (sometimes just one) can consume enough resources to add latency to the simpler queries. In contrast, the consumer product search query complexity is much more regularized and predictable.<br /></p><h2 style="text-align: left;">Precision vs. Recall</h2><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZMdUYMNcULvbviDIVpAhzWZDeSDCo9x-WFQk5fBCTc95EbDak447LoQQv2Dn4wGCHT5vszCEYlPKmXSGCsydC_6Z3mCkoJ43uWwjNwr8xcxlAM9XwoLj0xp2_Gtv3y3HiBTiy3tx0QW4W/s526/precision-recall.webp" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="340" data-original-width="526" height="163" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZMdUYMNcULvbviDIVpAhzWZDeSDCo9x-WFQk5fBCTc95EbDak447LoQQv2Dn4wGCHT5vszCEYlPKmXSGCsydC_6Z3mCkoJ43uWwjNwr8xcxlAM9XwoLj0xp2_Gtv3y3HiBTiy3tx0QW4W/w252-h163/precision-recall.webp" width="252" /></a></div><p></p><p>Suppose you have a product catalog containing 10,543 televisions and a user searches for the term "TV". Let's say your search engine happens to only match 10,327 for some reason. Is the user going to notice? Probably not. They have no idea how many products are in your catalog, or even the specific products in your catalog. The recall requirements here are a bit loose.</p><p>In contrast, in the legal domain, there is often a single document that is the "smoking gun" that can make or break a case. If an attorney searches for it and your search engine misses it in the search results, you will soon be out of business. If the document exists, your search engine *must* match it. <br /></p><p>The recall requirements are strict in the legal domain, but the precision requires are not. An attorney will not mind too much to sift through a few non-relevant documents. They are good at at pattern matching and filtering and would much prefer to get a few extra than to miss any.</p><p>Consumers search for products are a bit different. Showing a bunch of microwave ovens in your search results for a "TV" search is going to be a bad user experience if it happens to frequently. For product catalogs, trading off some amount of recall to improve precision is a good choice as it leads to "cleaner" search results for the user and they are none the wiser about what they might be missing. But in the legal domain, flawless recall is a requirement that is non-negotiable.</p><h2 style="text-align: left;">Data Normalization</h2><p> It is common for search engines to use "stop words" to filter out words with low semantic content, e.g., "the", "a", "an". Often this filtering is the default behavior. What the consumer product and legal domains share in common is that stopwords are a bad idea. There was a consumer brand name "THE" and an important legal case involving "Project A". Feed that data though a stopword filter and you lose some very important semantic information.</p><h2 style="text-align: left;">Technology<br /></h2><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMkwdOY8kkY_F5asq-VCT7LZQzPe4Lw7Nf_iM-6bF3Skdi4jWdUX8WGzxc0bn-rp4Gf2WHd7QaScyCtqYpidgYHeldI9g0CUjMlDxgLZYXZOWfysGy9ewZRazUzIwyl71UxDZ2_lJdbY6P/s474/lucene-logo.jpg" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="87" data-original-width="474" height="43" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMkwdOY8kkY_F5asq-VCT7LZQzPe4Lw7Nf_iM-6bF3Skdi4jWdUX8WGzxc0bn-rp4Gf2WHd7QaScyCtqYpidgYHeldI9g0CUjMlDxgLZYXZOWfysGy9ewZRazUzIwyl71UxDZ2_lJdbY6P/w235-h43/lucene-logo.jpg" width="235" /></a></div><p></p><p>All of the search technology I have used were based on Lucene. This includes Elasticsearch, SOLR and even a home-grown distributed version of Lucene back in the days before Elasticsearch existed. Lucene is an amazing piece of software and so is Elasticsearch. There is mostly no reason to use anything else. The only downside I found was in the legal domain where the data shapes and query shapes are atypical of most other domains. <br /></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyW_wYeP0zfHXMPHU6malVoXQ5ge2L22fcvLFuTF7KX_84bwhCK5fnxHnrDvV_IXU3dsOd6IiXTr4cAH7xN1jTc0sWxPbs63gWw7xTmJj6cc05peT0WZWgDSjNR0MdVMU5VrpW15saPkqR/s330/elasticsearch-logo.jpg" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="300" data-original-width="330" height="134" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyW_wYeP0zfHXMPHU6malVoXQ5ge2L22fcvLFuTF7KX_84bwhCK5fnxHnrDvV_IXU3dsOd6IiXTr4cAH7xN1jTc0sWxPbs63gWw7xTmJj6cc05peT0WZWgDSjNR0MdVMU5VrpW15saPkqR/w148-h134/elasticsearch-logo.jpg" width="148" /></a></div><p></p><p>Specifically, we ran into issues of limits and resource usage. Earlier versions of Lucene/Elasticsearch lack limits in some important code paths that resulted in run-away CPU or memory usage. As they have address these "holes" they have put limits in place that are too low to support some of the legitimate legal queries that need to be supported. Some thresholds are adjustable, but not all are, and there is always some amount of peril in playing with too many of the default parameters.</p><p>The Lucene and Elasticsearch teams focus on the most common cases for
query and data shapes, which is the right thing for them to do. Unfortunately, the legal domain use cases fall outside of
the norm. Consumer product searches, on the other hand, are more inline with their priorities.</p><h2 style="text-align: left;">Conclusion</h2><p>There are critically important differences in the search requirements for the two domains I have work in. It feels like these represent two extremes of a problem space, I wonder how many other domains match one of these two or if there are additional "classes" of search requirements with their own unique characteristics..<br /></p>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-32572269430580463412016-03-17T20:09:00.001-07:002021-01-31T23:13:16.892-08:00Product Normalization<p>Are the following two products the same?</p>
<table><tbody>
<tr><td>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhFqpSiLmGi6C7rYCItVr3_5cWscVYBaa0Zij6_3LDLZWtT8T-x2q6M2uNRdiBqu5jfsN6B7jsrxOKSU4Q-aadodJi6mUgU8k3BUW7_3Cc0RS9sL7TV5iMfKXv48SZFT2jCzqosQs8XXVV/s721/duracell-9v2-front.png" style="clear: left; display: block; float: left; padding: 1em 0px; text-align: center;"><img alt="" border="0" data-original-height="721" data-original-width="631" height="226" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhFqpSiLmGi6C7rYCItVr3_5cWscVYBaa0Zij6_3LDLZWtT8T-x2q6M2uNRdiBqu5jfsN6B7jsrxOKSU4Q-aadodJi6mUgU8k3BUW7_3Cc0RS9sL7TV5iMfKXv48SZFT2jCzqosQs8XXVV/w197-h226/duracell-9v2-front.png" width="197" /></a>
</td><td>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBWgCK0N-LyPSkfaFxlRl7YkfcB-KBCbKcNc3m71nQjogeA8kitvP2CZnWQaR7ddEpt2mIi3uAV779qfOZXcqL7U6MfrPyGEtw05jR8n2ugdOMIvrhl9zmI88yHSkcsrCJFrGAE3ivFUUd/s877/duracell-9v4-front.png" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="680" data-original-width="877" height="232" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBWgCK0N-LyPSkfaFxlRl7YkfcB-KBCbKcNc3m71nQjogeA8kitvP2CZnWQaR7ddEpt2mIi3uAV779qfOZXcqL7U6MfrPyGEtw05jR8n2ugdOMIvrhl9zmI88yHSkcsrCJFrGAE3ivFUUd/w299-h232/duracell-9v4-front.png" width="299" /></a>
</td></tr></tbody></table><p>
Do you think this is an easy question to answer or a hard one? The correct answer is: "it depends" and the bigger part of that is "it depends on how you define the word 'product'".</p><p>I have worked on this problem, called "product normalization", for more than 5 years and at two different companies. Not only was I required to be able to answer questions like these, but I needed to encode the "rules" for doing this and build software systems that could make these decisions automatically (and at scale).</p><p>Before diving any deeper, if you believe the answer to the question is either "yes" or "no", let me try to convince you that the answer is not so cut-and-dry. <br /></p><p>The first search-based company I worked for was a comparison shopping company. It was a site where you could find the best price for a product by comparing all the store prices in one place. In this context, those two products above are definitely *not* the same. One of them is going to cost twice as much, so if we showed the price for the "2-pack" from one store next to the price for a "4-pack" from another store, we are not providing the user with a meaningful comparison experience.</p><p>The next company I worked for dealt with managing customer reviews. Suppose there are some meaningful reviews on the 2-pack and none for the 4-pack, but the user is looking at the 4-pack product page. Do we show the 2-pack reviews on the 4-pack page? In this case, the answer is "yes", we would not want the user to miss out on relevant reviews about the quality of the product.<br /></p><p>Therefore, when we are comparing price, the quantity in the package is very important, and these should not be considered the same. But when trying to make buying decision based on the quality and features of the product, the quantity is not important.</p><h2 style="text-align: left;">The Fallacy of UPC matching</h2><p>The most basic way to compare products for "sameness" is to look at their UPC codes (EAN codes for the Europeans). As illustrated above, this is not going to help if you are matching for the purpose of consolidating product reviews. The 2-pack and 4-pack surely have different UPC codes. </p><p>But suppose we only need to solve the problem in the price comparison domain. Comparing UPC codes will tell us immediately these are not the same. This starts to make the price comparison case seem trivial to solve: just match on UPC codes.</p><p>This would seem to work great, though you have to assume the following:</p><ul style="text-align: left;"><li>you have all the UPC data for all the products; and<br /></li><li>the UPC data is accurate for all your products.</li></ul><p>However, there are some harsh realities about UPCs that really complicate things:</p><ul style="text-align: left;"><li>assumption of completeness and accuracy of data are always highly dubious; <br /></li><li>not all products are assigned UPCs (e..g, clothes);</li><li>some products are assigned more than one UPC;</li><li>UPC values can be re-used for completely different products;</li><li>trivial changes to a product can result in a new UPC (e.g., sometimes just a different packaging causes a new UPC to be used).</li></ul><p> We can illustrate further with a final example. Are these two products the same?</p>
<br />
<table><tbody>
<tr><td>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2ZcYmWkcYhxicQSf_-K4_e1mj_06jKDlfxS_hWhCtRROa-OLnZSeiv91ai-sx_683u9OVtF6sLFyCIveis39fcmOuE2ix5OD1X8uSozq7m8nKj3AVOzXLWDs9jhyJFKhwx026UhXOVbna/s877/duracell-9v4-front.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="680" data-original-width="877" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2ZcYmWkcYhxicQSf_-K4_e1mj_06jKDlfxS_hWhCtRROa-OLnZSeiv91ai-sx_683u9OVtF6sLFyCIveis39fcmOuE2ix5OD1X8uSozq7m8nKj3AVOzXLWDs9jhyJFKhwx026UhXOVbna/s320/duracell-9v4-front.png" width="320" /></a>
</td><td>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjq_utHbHQ46VqXmtsUMHrQDzA-f3onBOBBWNQ0FH-0RJVtj66x0dLHW68nYihs66nV4jP9IJ5TtJkvUP3Rdt0dbCyNp9J51xZfWHoBXpJroAhMRncGbjtYkm1yMpKULKrJNY2CA5yUQPQ9/s735/duracell-9v4-alt-front.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="735" data-original-width="683" height="253" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjq_utHbHQ46VqXmtsUMHrQDzA-f3onBOBBWNQ0FH-0RJVtj66x0dLHW68nYihs66nV4jP9IJ5TtJkvUP3Rdt0dbCyNp9J51xZfWHoBXpJroAhMRncGbjtYkm1yMpKULKrJNY2CA5yUQPQ9/w235-h253/duracell-9v4-alt-front.png" width="235" /></a>
</td></tr></tbody></table><p>
<br />
The quantity is the same, they are now both 4-packs, but the packaging is slightly different. </p><p>The answer would seem to be "yes, they are the same" for both the price comparison and the product review cases. However, flip the packages over and you will notice they have completely different UPC values: </p><p><br />
</p><table><tbody>
<tr><td>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3_9hblRBmUUJzOt-TPaY_AyODeXSpLrvdsLk1ymI1JnURaYCgUQuSbflaygbryDmligODYu4X2p9UGuIhaFHhV0o-LdCKzWsmkjD1GyaUSLLLdLl1opcMwXgfgdvfBnIjaQnJs4M8olck/s869/duracell-9v4-back.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="670" data-original-width="869" height="252" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3_9hblRBmUUJzOt-TPaY_AyODeXSpLrvdsLk1ymI1JnURaYCgUQuSbflaygbryDmligODYu4X2p9UGuIhaFHhV0o-LdCKzWsmkjD1GyaUSLLLdLl1opcMwXgfgdvfBnIjaQnJs4M8olck/w327-h252/duracell-9v4-back.png" width="327" /></a>
</td><td>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZfkMxhBcBlsxHULJhwJ-MZ23h0nzgSzPBlyoSAU9DxlztGsHJl-31J3c8S40RsIFsZiUybTPCRIJzrNvzSfloekfELfsHW6PrJJXM8KXOI-YSW4vQ0MpAJ07DQtaqzkIM_7EaE6FsEi5X/s726/duracell-9v4-alt-back.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="726" data-original-width="633" height="252" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZfkMxhBcBlsxHULJhwJ-MZ23h0nzgSzPBlyoSAU9DxlztGsHJl-31J3c8S40RsIFsZiUybTPCRIJzrNvzSfloekfELfsHW6PrJJXM8KXOI-YSW4vQ0MpAJ07DQtaqzkIM_7EaE6FsEi5X/w219-h252/duracell-9v4-alt-back.png" width="219" /> </a></td></tr></tbody></table><p>
<br />Hopefully this gives you some introductory flavor for the complications of product normalization. I've written a much more in-depth and technical document on this subject. It's a whopping 60 pages, so not something to read unless you really need to solve this problem and need some ideas of approaches. You can read it here:</p><ul><a href="https://docs.google.com/document/d/17_UB6b3HsazlfBY5nOekhfonlIeCm4HlJxU1vKCX-9g/edit?usp=sharing" target="_blank">Product Normalization White Paper</a></ul>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-21020342095581700762013-02-09T12:24:00.001-08:002013-02-09T20:53:16.694-08:00QUnit Inside Firefox Extensions<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgihw_w5hYiQ47CCJvjbyhrqTFr5LcQaeAWKjwZosTQLFPqUIxi6hWCkOBTIVulEns0Rt6AfBsXgjKtv8bBTuqouNbgaKn6Pa7-La5BU8rWgFTJsn4xS8YAvOG4NHpn9AMX8CTIF0Vgij8I/s1600/qunit-chrome.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgihw_w5hYiQ47CCJvjbyhrqTFr5LcQaeAWKjwZosTQLFPqUIxi6hWCkOBTIVulEns0Rt6AfBsXgjKtv8bBTuqouNbgaKn6Pa7-La5BU8rWgFTJsn4xS8YAvOG4NHpn9AMX8CTIF0Vgij8I/s320/qunit-chrome.png" width="320" /></a></div>
<br />
<br />
<br />
<br />
I have a fairly large amount of code inside one of my Firefox extensions (<a href="https://addons.mozilla.org/en-US/firefox/addon/shopping-helper/" target="_blank">Shopping Helper</a>) . I also have always manually tested this code, which made changes hard to do. I recently did a major refactoring of the code, and have plans to add more features, so it was time to get serious about automating the testing, with the first step of adding a comprehensive unit test suite. Now, just about every language has a useful framework, usually called something like "FooUnit" (e.g., JUnit, PyUnit), so that's the sort of thing I wanted for my extension.<br />
<br />
<br />
Here's the problem: Firefox extensions are JavaScript, but it runs in a very different context that the normal JavaScript inside an HTML page situation. This is typically called "chrome code", to differentiate it from the code that runs in the HTML document itself. This alternate run-time environment has always made debugging and development tricky for Firefox extensions. It gets further complicated by security issues where the chrome code is trusted more than code inserted into HTML, so there is a "wall" between the two worlds.<br />
<br />
Looking for JavaScript unit test frameworks, <a href="http://qunitjs.com/" target="_blank">QUnit</a> seemed to be exactly what I was looking for: same basic framework as the FooUnits, relatively mature and well regarded. It is nicely done, but it is also not done for the purposes of being used as chrome code inside a Firefox extension. The question is whether it would run as-is inside chrome code (very doubtful) or could be made to run inside chrome code (more hopeful) without too much effort (a bit worrisome).<br />
<br />
Short answer is that QUnit does not run as-is in the chrome context, but can be made to do so with a moderate amount of work (it took me about 5 hours). Since I have now done this work, no one else needs to do this, so for you, it will be relatively easy with all the details I give below.<br />
<br />
First off, here are the files you need, which are based on version 1.11.0 of QUnit.<br />
<ul>
<li><a href="http://www.cassandra.org/projects/computer/qunit-chrome-1.11.0.js" target="_blank">qunit-chrome-1.11.0.js</a></li>
<li><a href="http://www.cassandra.org/projects/computer/qunit-1.11.0.css" target="_blank">qunit-1.11.0.css</a></li>
</ul>
Below are the details of the changes I had to make and how to go about using this inside Firefox chrome code.<br />
<h2>
</h2>
<h2>
QUnit Javascript Changes</h2>
<h4>
</h4>
<h4>
Chrome vs. Non-chrome Control</h4>
<br />
To not have to fork the QUnit code for the modifications, I defined a global variable <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">QU_ISCHROME</span></span> whose setting can be changed to easily switch to being used inside a Firefox extension in a chrome window instead of a browser window. This will default to 'false' to give the previously existing behavior and can be changed to 'true' when including in the XUL/chrome Firefox extension context. It should be a simple one value change to get the effect, though I did not check if it ran outside a XUL window.<br />
<h4>
</h4>
<h4>
Altering <span style="font-family: "Courier New",Courier,monospace;">createElement()</span> calls</h4>
<br />
Since QUnit creates HTML DOM nodes, and since it will live inside an XML/XUL window, when creating these nodes, we have to explicitly define the namespace when creating them because the XUL namespace is not the standard HTML tags. We do this by replacing all calls like this:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> document.createElement( "tag" );</span></span><br />
<br />
with something like this:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">document.createElementNS( QU_HTMLNS, "html:tag" );</span></span><br />
<br />
where '<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">QU_HTMLNS</span></span>' is a conveniently defined global variable we put at the top of the Javascript file which is defined as:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">var QU_HTMLNS = "http://www.w3.org/1999/xhtml";</span></span><br />
<br />
We add a convenience function <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">QU_createElement()</span></span> and wrap all node creation in it to keep the code cleaner.<br />
<h4>
</h4>
<h4>
Altering <span style="font-family: "Courier New",Courier,monospace;">innerHTML</span> Content Assignments</h4>
<br />
While doing an assignment of an HTML fragment to a node via the <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">innerHTML</span></span> property works fine in a browser window, this does not work well in a chrome window. Thus, we wrap this assignment in a helper function <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">QU_setInnerHTML()</span></span> so that when <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">QU_ISCHROME</span></span> is true, it uses a proper parser directly.<br />
<h4>
</h4>
<h4>
Altering <span style="font-family: "Courier New",Courier,monospace;">innerHtml</span> Content Fetching</h4>
<br />
Similar to the assignment problem of<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> innerHTML</span></span>, using it as a value does not work that will in the chrome context either, so we also wrap these in a helper function <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">QU_getInnerHTML()</span></span>.<br />
<h4>
</h4>
<h4>
Pedantic QUnit Javascript (optional)</h4>
<br />
In a more pedantic mode, a number of <b><span style="font-size: x-small;">J</span></b>avaScript warning were generated around functions that did not always return a value and properties that were used but not defined.<br />
<h2>
</h2>
<h2>
Extension Development Environment Notes</h2>
<br />
I package up my extension with a bash script that conditionally includes testing and debug code based upon a command line parameter. So all the QUnit code is not in the production build. This requires a bit of a customized extension environment though one could just manually add and remove the <a href="http://qunitjs.com/" target="_blank">QUnit</a> code if you did not want to have to replicate conditional building. When included by the build script, the code also insert a menu option to allow invoking the unit test window.<br />
<h2>
</h2>
<h2>
XUL Window Notes</h2>
<h4>
</h4>
<h4>
CSS Include</h4>
<br />
Need to have the CSS includes after the XML declaration, but before the window itself.<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> <?xml-stylesheet href="chrome://myextension/skin/qunit-1.11.0.css"<br /> type="text/css"?></span></span> <br />
<h4>
</h4>
<h4>
Namespace include</h4>
<br />
The window XML object needs the attribute to ensure that HTML elements can be put inside it:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> xmlns:html="http://www.w3.org/1999/xhtml"</span></span><br />
<h4>
</h4>
<h4>
JavaScript include</h4>
<br />
Within the window XML, include the QUnit JavaScript. Here I have renamed it to reflect that it is the version with the <span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">QU_ISCHROME=true</span></span> setting:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> <script type="application/x-javascript"<br /> src="chrome://shophelper/content/qunit-chrome-1.11.0.js"/></span></span><br />
<h4>
</h4>
<h4>
Body Elements</h4>
<br />
As with the normal QUnit usage, you need the special two "<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">div</span></span>" elements. The twist here is that the HTML tag names need to be preceded with "<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">html:</span></span>"<span style="font-size: x-small;"> </span>to explicitly define their namespace.<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> <html:div id="qunit"></html:div><br /> <html:div id="qunit-fixture"></html:div></span></span><br />
<h4>
</h4>
<h4>
Full Example XUL Window</h4>
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"><?xml version="1.0"?><br /><br /><?xml-stylesheet href="chrome://global/skin/" type="text/css"?> <br /><?xml-stylesheet href="chrome://myextension/skin/qunit-1.11.0.css" type="text/css"?> <br /><br /><window id="MyExtensionUnitTestDialog"<br /> xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul"<br /> xmlns:html="http://www.w3.org/1999/xhtml"<br /> onload="SHUTD_onLoad(event);"<br /> ondialogcancel="return SHUTD_onDialogCancel();"<br /> title="My Extension Unit Test Console"<br /> buttons="cancel"><br /><br /> <script type="application/x-javascript"<br /> src="chrome://myextension/content/qunit-chrome-1.11.0.js"/><br /> <script type="application/x-javascript"<br /> src="chrome://myextension/content/unit-test-dialog.js"/><br /> <br /> <vbox flex="1"><br /> <box><br /> <html:div id="qunit"></html:div><br /> </box><br /> <box><br /> <html:div id="qunit-fixture"></html:div><br /> </box><br /> </vbox><br /></window><br /> </span></span><br />
<h2>
XUL (extension) Javascript Notes</h2>
<br />
To test functions in the main extension within the QUnit XUl windo, we need access to the main window where the extension chrome code runs. I am not sure if there are better ways to do this, but this works by iterating over windows.<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> var parentWindow = null<br /> var wm = Components.classes["@mozilla.org/appshell/window-mediator;1"].getService(Components.interfaces.nsIWindowMediator);<br /> var enumerator = wm.getEnumerator("navigator:browser");<br /> while(enumerator.hasMoreElements()) <br /> {<br /> var win = enumerator.getNext();<br /> if ( ! win.document )<br /> continue;<br /> parentWindow = win;<br /> break;<br /> }</span></span><br />
<br />
You then can invoke functions from the extension with:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> parentWindow.someFunctionName( someArg1, someArg2 );</span></span><br />
<h2>
</h2>
<h2>
QUnit CSS Changes (optional)</h2>
<br />
Inside the chrome windows, the CSS property "<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">-moz-border-radius</span></span>" does not apply, so it gives a warning.archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-25936194272210556702012-08-01T16:43:00.000-07:002024-03-07T20:15:33.021-08:00Ignorant Juries - By Design<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTrb7-eQW8vCzbm6riRFk2XMeyzRVhHaXXnYAvVWNeyUFlG-1nvGy-Mx07z3WkQVBI0msOr0GOrxf4NxIJACiuK8JGRW_7kBBOPk1WCiQt-tT5GrX7fkQ4UFuEp8s_hNbvkIlrI-OJEAeil5hVdLtkCLVGJ-EEM9rmH0balaEq3W2HZrkz9HPg3TIgHiPH/s1024/DALL%C2%B7E%202024-03-07%2022.15.04%20-%20Envision%20a%20courtroom%20scene%20where%20the%20jury%20box%20is%20filled%20with%20thirteen%20monkeys%20dressed%20as%20people,%20arranged%20in%20two%20rows%20to%20reflect%20a%20traditional%20jury%20se.webp" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="1024" data-original-width="1024" height="285" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTrb7-eQW8vCzbm6riRFk2XMeyzRVhHaXXnYAvVWNeyUFlG-1nvGy-Mx07z3WkQVBI0msOr0GOrxf4NxIJACiuK8JGRW_7kBBOPk1WCiQt-tT5GrX7fkQ4UFuEp8s_hNbvkIlrI-OJEAeil5hVdLtkCLVGJ-EEM9rmH0balaEq3W2HZrkz9HPg3TIgHiPH/w285-h285/DALL%C2%B7E%202024-03-07%2022.15.04%20-%20Envision%20a%20courtroom%20scene%20where%20the%20jury%20box%20is%20filled%20with%20thirteen%20monkeys%20dressed%20as%20people,%20arranged%20in%20two%20rows%20to%20reflect%20a%20traditional%20jury%20se.webp" width="285" /></a></div><div>I was reading <a href="http://www.itworld.com/it-managementstrategy/288055/apple-v-samsung-how-it-was-explained-jury" rel="nofollow" target="_blank">this article</a> about a judge prepping a jury for a patent infringement case between Apple and Samsung. Given the horrendous state of the U.S. patent system, it seems to me that the more you understood about software patents, the more biased you would be and thus much less likely to ever be selected for this jury. Additionally, the more you understood of the creative process, which really starts by copying (and then tweaking), the more biased you would be against design patents, Thus, it seems that the people most ignorant, and least qualified on the topic of patents are the ones deciding the verdict.</div>
<br />
This made me realize that this phenomenon is probably not limited to software patents. In any field, the more informed you are about the topic, the more likely you would be labeled as biased toward a case. Therefore, our judicial system would seem to be driven by those least qualified to make judgements.<br />
<br />
<br />
To keep it in perspective though, what is likely a far worse problem is when attorneys choose jurors they think are the most malleable, rather than most qualified. The judicial process then boils down to which attorney is the better salesperson and not whether there is actual guilt or innocence. <br />
<br />archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-31105595225796493282012-07-25T18:58:00.000-07:002012-07-25T18:58:13.065-07:00License Plate Sanity Restored<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUsY6GgLgr5s5xU_TuVc6wUvsvyRDdkEa1dN9xDoB7mi2Q7nfXeES9n_ve8g5jMfsZ6e2XNKeKmi-T3Z1auYqR2WrAwjeev0-JlNRXc1Kv84rI-6RtOIiNXmWpVASJgt8Yvm0XZX_IRXbj/s1600/tx-plate-old.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="160" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUsY6GgLgr5s5xU_TuVc6wUvsvyRDdkEa1dN9xDoB7mi2Q7nfXeES9n_ve8g5jMfsZ6e2XNKeKmi-T3Z1auYqR2WrAwjeev0-JlNRXc1Kv84rI-6RtOIiNXmWpVASJgt8Yvm0XZX_IRXbj/s320/tx-plate-old.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Stupid Design</td></tr>
</tbody></table>
About 3 years ago, Texas changed their license plate design to something
that made them unreadable from more than 10 feet away. I am sure they
looked beautiful in a high-resolution, brightly lit setting to a
committee of clueless overseers, but they failed to deliver on what
should have been the number one requirement: readable at a distance in a
variety of lighting conditions.<br /><br />How does a group of people get
put in charge of redesigning license plates and fail to deliver on the
readability requirement? How is is that at every step of the design and
approval process no one ever raised this question? Did they ever even
think to consult the number one user of the plates: law enforcement?
Would any police officer fail to spot this fatal design issue
immediately? This was an epic failure of government, committees and
common sense.<br /><br />It may not sound like it, but this is meant to be a
positive, uplifting story. We come to that end by noting that Texas has
corrected the problem and is (yet again) going to put out a new design,
precisely to address the current problem of readability. And it took
the Texas government only 3 years to recognize and fix the problem:
maybe a new record.<br /><br />This new design harken back to much older
designs, and I like them a lot. Simple, effective and functional. Now I
have to figure out how to trade in my plate for a new style one.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoENJtzbWknGuD-Bq6Gh5J4nLfPCJ3Fhb5QjBPzQiKhgbD8qElfO3iEVxYsusFkQiy9xNla5KPBnwgX6i64dtS6CWUcANwBtiM1iB4KF27rzDaNnrRNUdSJ_xQwrKVvSOL3TiuZ1ckpmI0/s1600/tx-plate-new.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="154" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoENJtzbWknGuD-Bq6Gh5J4nLfPCJ3Fhb5QjBPzQiKhgbD8qElfO3iEVxYsusFkQiy9xNla5KPBnwgX6i64dtS6CWUcANwBtiM1iB4KF27rzDaNnrRNUdSJ_xQwrKVvSOL3TiuZ1ckpmI0/s320/tx-plate-new.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sensible Design</td></tr>
</tbody></table>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-83027867023375513542012-07-17T18:05:00.000-07:002024-03-07T20:11:53.152-08:00Elevator Insanity<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOcq2KhvbkcLmazlyec2dFiVyf9itBZ3Xx6OWMUQNkhiv-E7xWnNIDQ3j5IG7u7gk56TjK9GeUm9qf3p_w4LOJ0-8ysOH_2Ytc7YkR73C-YtuRAdC9iInk6F6HFOifOl5riXH33tKLZTScwRmHse3FUclhU7m9C3wFEvl1PATKkiEXZ_iijD53-at5qKez/s1024/DALL%C2%B7E%202024-03-07%2022.11.13%20-%20Visualize%20an%20elevator%20control%20panel%20that%20is%20unusually%20perplexing%20and%20chaotic.%20This%20panel%20is%20filled%20with%20buttons%20that%20are%20not%20in%20any%20logical%20order,%20fea.webp" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="1024" data-original-width="1024" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOcq2KhvbkcLmazlyec2dFiVyf9itBZ3Xx6OWMUQNkhiv-E7xWnNIDQ3j5IG7u7gk56TjK9GeUm9qf3p_w4LOJ0-8ysOH_2Ytc7YkR73C-YtuRAdC9iInk6F6HFOifOl5riXH33tKLZTScwRmHse3FUclhU7m9C3wFEvl1PATKkiEXZ_iijD53-at5qKez/w274-h274/DALL%C2%B7E%202024-03-07%2022.11.13%20-%20Visualize%20an%20elevator%20control%20panel%20that%20is%20unusually%20perplexing%20and%20chaotic.%20This%20panel%20is%20filled%20with%20buttons%20that%20are%20not%20in%20any%20logical%20order,%20fea.webp" width="274" /></a></div><div>Having been in NYC the last 10 months, I have encountered more elevators than I previously have in the past. I have grown to have some contempt for elevator manufacturers. I just do not understand why the very small number of manufacturers could not agree on some standards. You have your "G", which could mean "ground floor" or it could mean "garage". You have the "L" which could mean "Lobby" or "Lower". Then there's the "M", which could mean "mezzanine" or it could mean "main floor". I have even seen custom letters in some hotels that stand for some special feature on that floor, which of course has no meaning unless you know the secret code. Is "R" a "roof" or a "restaurant"? Do they dare use "G" for "gym"? I do not care what letter they use for what, just the consistency would stop me from pressing the wrong button as I go from one building to the other.</div>
<br />
Then there is the inconsistency about where floor numbering starts. Sometimes the first floor it is the same as "main", "lobby" or "ground" floor, but not always. So if you see a "G" and a "1", you really have no idea what is what. The one thing that does add a little bit of sanity is the "star" on the button to indicate the "main" floor to get off (which could be "M", "G" , "L"or "1"). For simple buildings with one exit, this is usually satisfactory, but there are a lot of places with multiple levels of exits and one usually does not know exactly what level they entered on. You can enter from the street and not be on the "ground" floor, or you can enter in way where you have no idea what level you are on. In those circumstances, how I am supposed to know what they consider the "star" level or exactly what button I need to exit the way I came in?<br />
<br />
And my biggest gripe comes from the elevator in my apartment building. The buttons go from "1" to "16", with a"star" on the "1". All this is perfectly good and unambiguous. And what would you expect the LED readout to show when you reach floor "1"? You would expect to see "G" naturally. They could not even stay consistent within a single elevator design.<br />
<br />archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-21257244350225355932010-01-31T11:37:00.001-08:002010-01-31T11:41:40.757-08:00Shopping Helper UpdatedMy Firefox add-on, Shopping Helper, has been updated and improved. It's a great tool for helping people shop on-line. You can keep shopping lists and favorite stores and it automatically keeps track of the prices, allowing you to view products in a comparison grid, and alerting you when there is a significant price change. You can get it here:<br /><br /> <a href="https://addons.mozilla.org/en-US/firefox/addon/7942">https://addons.mozilla.org/en-US/firefox/addon/7942</a><br /><br />And visit the home page for overview webcasts, screenshots and help:<br /><br /> <a href="http://labs.pronto.com/public/shopping-helper/index.html">http://labs.pronto.com/public/shopping-helper/index.html</a>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-76156350523362326032009-05-31T18:54:00.000-07:002009-05-31T19:11:20.500-07:00Google Wave ThoughtsI've read a bunch of hype about the Google Wave presentation and then actually watched the hour+ long presentation that the Google Wave team gave. Here's some thoughts on the matter.<br /><br />Is this truly the revolutionary architecture they are hyping? It could be, but only if people buy into it and it does not wind up being too complicated to understand. Thus, its success depends on how well it is hyped, how well they can manage the early-adopted/tech crowd and if they can have enough stuff built on top of it for the average person to get some uptake. Given that, the early developer preview demo and release makes a lot of sense.<br /><br />The basic fact is that they have the right idea: email is archaic, IM is archaic, and there are all these emerging new interaction patterns: Twitter, Facebook, blogs, etc. Can't we put all these things under one roof? My answer is: maybe. If you design something too general, it fails for being too complex/abstract to comprehend. I wonder if the generality of Google Wave may suffer from trying to be too general.<br /><br />In some sense, you could give a demo of the TCP/IP stack today and woo the crowd with all the things you could do with it: email, IM, blogs, facebook, twitter, etc. Wow, this thing is amazing!<br /><br />So is it something the user cares about or something only developers care about? I think it is too complex/abstract for the end-user to consider or care about. But maybe we sit in a time where something as general as TCP/IP is needed, only one level up, and maybe this fits the bill. There's no question that building apps today is a royal pain in the butt, especially handling concurrent events and all that makes up implementing the more collaborative things. If Google Wave winds up being the API that makes all this easier for the developer, maybe it will get uptake and be the "next big thing".archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-90832765520660784112008-07-16T20:08:00.000-07:002008-07-16T20:10:45.265-07:00Followup 2: Shopping HelperThere's now an on-line video demonstration/tutorial for the Shopping Helper Firefox add-on:<br /><br /> <a href="http://www.youtube.com/watch?v=EXBgdo6ktmw&fmt=18">Watch on YouTube</a>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-44938109508810958192008-07-02T21:56:00.000-07:002008-07-02T22:01:22.069-07:00Followup: Shopping HelperThe Firefox Add-on to help you shop on-line that I mentioned in a previous blog entry has now started to go through the process of being hosted on Mozilla's addons site:<br /><br /> <a href="https://addons.mozilla.org/en-US/firefox/addon/7942">Shopping Helper Page on Mozilla's Add-ons Site</a><br /><br />It is still in their sandbox as an experimental add-on at the time of this writing, but if you don't want to have to log in to download it, you can still get it here:<br /><br /> <a href="http://labs.pronto.com/public/shopping-helper/index.html">Shopping Helper Page on Pronto.com Labs Site</a>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-46994541208990241032008-05-30T18:01:00.000-07:002008-05-30T18:06:31.670-07:00Shopping Helper Firefox Add-onIf you shop a lot on-line, there's a neat Firefox add-on that is coming out that offers a lot of useful features. It's Pronto's Shopping Helper, which allows you to create shopping lists, get price alerts, and view current prices. It has a ton of other minor features, but is overall a useful tool for organizing and tracking products you hope or plan to buy some day. You can see the preview here, which will also eventually point you to an actual download (should be within the next few weeks).<br /><br /> <a href="http://labs.pronto.com/public/shopping-helper/index.html">Pronto's Shopping Helper Firefox Add-on<br /></a>archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-15756968162398142742008-04-03T17:20:00.000-07:002024-03-07T20:17:31.309-08:00Absurb Baseball Ticket Prices<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjq_alwyDDlKZ-KVsqi1rsZx0iTQOMyQz6sbo2TwkwTZzETZzvImY3VzSFEzY6pGIlRZWZzmgjbxIELgvkAw9Ph1o9x27RJGeueid6LnlH-q6t5cmnpbD6eiF0VQJyRMHOrTxTJEkivjJRnDJLigpMEryZ0gH8r2rJXc-cs3PT9Qr7paeUwsRTJFP7DReTe/s1024/DALL%C2%B7E%202024-03-07%2022.16.45%20-%20Design%20a%20realistic-looking%20ticket%20to%20a%20baseball%20game,%20featuring%20detailed%20elements%20like%20team%20logos,%20seating%20information,%20and%20the%20date%20of%20the%20event.%20The.webp" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="1024" data-original-width="1024" height="285" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjq_alwyDDlKZ-KVsqi1rsZx0iTQOMyQz6sbo2TwkwTZzETZzvImY3VzSFEzY6pGIlRZWZzmgjbxIELgvkAw9Ph1o9x27RJGeueid6LnlH-q6t5cmnpbD6eiF0VQJyRMHOrTxTJEkivjJRnDJLigpMEryZ0gH8r2rJXc-cs3PT9Qr7paeUwsRTJFP7DReTe/w285-h285/DALL%C2%B7E%202024-03-07%2022.16.45%20-%20Design%20a%20realistic-looking%20ticket%20to%20a%20baseball%20game,%20featuring%20detailed%20elements%20like%20team%20logos,%20seating%20information,%20and%20the%20date%20of%20the%20event.%20The.webp" width="285" /></a></div><div>I guess it has been a long time since I've paid attention, but a recent exploration into buying a couple tickets to an LA Dodgers home game was quite the experience.</div><br /><span style="font-weight: bold;">Too Many Choices and Stupid Names</span><br /><br />Back "in the day", you had your box seats, your mezzanine, your upper deck and your outfield bleachers: 4 choices with meaningful names. At Dodger stadium there are <a href="javascript:gener_seatven('/seatingchart/90145/21986','eventd')">22 different names</a> for areas of the ballpark, none of which really tell you anything about where they are located. Where would you expect "Infield Reserve" to be? Well it is the upper deck area, though it is closer to the infield that the other upper deck areas. "Lower Reserve"? It's actually upper deck, but not as upper deck as some other seats.<br /><br />Even when they have a meaningful part in their name, they add these cutesy variants that again give you no idea what the difference is. There's "Premium", "VIP" and "MVP", all of which tell you nothing about where they might be located relative to one another.<br /><br /><span style="font-weight: bold;">The Color Eye Chart Test</span><br /><br />More brilliant graphic design here in the seating chart. Those 22 sections each have a unique color that you have to match up against the key that labels the sections. Each primary color has 4 to 5 different shades, some of them impossible to distinguish when trying to cross-reference with the diagram. <a href="http://en.wikipedia.org/wiki/Edward_Tufte">Edward Tufte</a> would be proud.<br /><br />Hunting around to write this blog entry, I did find a <a href="http://losangeles.dodgers.mlb.com/la/ballpark/la_seating_pricing.jsp">better location</a> to view they seating chart. Seems I got to a Ticketmaster site when going from their schedule to buy tickets. Guess I should have known that was not the right way to do things. However, this better chart page suffers form the typical MLB pages that seem to assume you have 1 Gigbit bandwith to their servers.<br /><br /><span style="font-weight: bold;">The Punch Line</span><br /><br />I was debating whether to get the cheapest seats, just to get into the park, or maybe splurge $30 a piece to get an upgrade. With the price ranges at Dodger stadium, $30 does not get you much, and in fact, I think they get you worse seats than the bleachers.<br /><br />In their ticket buying options (again, this is really a Ticketmaster site), they have a 'find best available' choice. So I am curious as to what might be the best available so I could make a decision on whether they were worth it or not. What they suggest to me, are the "Dugout Club" seats at $500 a piece....$500 a piece....you've got to be kidding me.<br /><br />They are not even the best seats for watching the game, though I suspect anyone who thinks a $1,000 for two baseball tickets probably is not really caring about getting the best "baseball watching" seats. They just want to be exclusive, and have 40,000 other people around them to observe their exclusiveness.<br /><br />Professional baseball is doing everything they can to drive away baseball fans. The prices, the catering to non-baseball fans, the greed, everything but the game is the focus. MLB is an evil organization and it is painful to watch what they have done to a great sport.archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-65718468072611820222008-03-26T21:09:00.000-07:002008-03-26T21:21:12.391-07:00The Semantic WebWhat is the "Semantic Web"? It's nothing.<br /><br />No one knows how to define it other than just listing about every current web-related technology being developed. It's at best a catch-all for the current technology, and at worst a meaningless term that a bunch of charlatans use to convince someone to give them money, or to convince others they are "technically hip".<br /><br />Some day in the future, something being developed today will be useful, then someone with great hind-sight will say "Yes, that is what I meant by the Semantic Web!"<br /><br />Keep in mind, you <span style="font-style: italic;">cannot</span> encode semantics in any computer representation. Anyone whose says you can is kidding you and/or themselves. It's all just syntax, since semantics are something no one really understands other than some concept people seem to agree on.<br /><br />If I say "car", you know what I mean and I know what I mean. Since we likely agree, or can find no evidence that we are not referring to the same concept, the three characters can be said to have semantic meaning, but they do not really. It is you and I that give those characters semantics, not the computer. Most work on semantics and ontologies just create more elaborate syntax, which generally just obfuscates things so people <span style="font-style: italic;">think</span> its semantics, and even probably convince themselves it is.archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-70061833101516596652008-02-04T12:05:00.000-08:002012-08-01T16:47:44.080-07:00Superbowl Upset?The Giants and Patriots played each other 5 weeks before they met in the superbowl. The game was very competitive, with the Pats winning by just 3 points. Given these facts, can someone explain to me:<br />
<ol>
<li>why the entire sports journalism world is calling this an upset; and even more ridiculously</li>
<li>why there is even debate on where this sits as one of the greatest upsets of all time?</li>
</ol>
I guess the answer is that logic, and prior football knowledge is secondary to outcomes that simply disagree with the media-induced frenzy that is the superbowl hype-machine.archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-9661018433536405262008-01-13T08:02:00.000-08:002008-01-13T08:11:48.092-08:00Shameless JournalismThis morning's top "news" story on CNN:<br /><br /><blockquote><a href="http://money.cnn.com/2008/01/13/autos/car_of_the_year/index.htm?postversion=2008011310"><span style="font-weight: bold;">Chevrolet Malibu wins Car of the Year</span></a><br /><br />General Motors' Chevrolet Malibu mid-size sedan, radically redesigned for the 2008 model year, won the North American Car of the Year Award today at the first media preview day for the Detroit Auto Show.<br /></blockquote><br />This is not news, but free advertising for General Motors. Just more signs that corporations, one way or another, are deeply connected to the media and are being used to manipulated the public. And if you think that being the "car of the year" is generally interesting news to people, I might agree<span style="font-style: italic;"> if</span> this was really the story. But if you look carefully, it is qualified as the "North American Car of the Year", which implies it really is not the car of the year since they've excluded all the imports that typically win the real car of the year awards. Maybe CNN is just trying to show their patriotic side?archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0tag:blogger.com,1999:blog-579442270548217340.post-72883570530736601482008-01-08T19:59:00.000-08:002008-01-08T20:07:33.548-08:00Goose Gossage and Hall of FameFrom <a href="http://www.sportsline.com/mlb/story/10565288">cbssportsline.com</a> this evening:<br /><blockquote>"It's insane it took nine years on the ballot for Goose Gossage to <a href="http://www.sportsline.com/mlb/story/10565288">get elected</a> to Cooperstown, but thankfully voters come to their senses."<br /></blockquote>Kind of funny to see a sportswriter say that what sportswriters do is "insane" (it is sportswriters that determine who gets in the hall of fame after all). Anyone paying attention knowns that (almost all) sportswriters are definitely not journalists, and are more typically not any smarter or thoughtful than a random person off the street. So when a sports media outlet publishes a headline that effectively admits that sportswriters, as a group, are behaving with no rhyme or reason (or possibly with deliberate capriciousness), that's something I find refreshing.archttp://www.blogger.com/profile/03694523494127499522noreply@blogger.com0