Windows 10's "Notify to schedule restart" option is evil

Happy Patch Tuesday, everyone.  This is actually my first Patch Tuesday on Windows 10, and I'm learning some interesting things about how it works in 10.

By default, Windows Update will reboot the computer automatically when it's not in use.  That's fine for most people, but since my computer is essentially a gaming rig, I don't want to trust Microsoft on the whole "when it's not in use" thing.  I have occasionally seen other software vendors screw this up, and I'd rather choose reboot times myself.  I rarely leave the computer on for more than 12 hours on end, so there is usually no need to reboot the computer anyway.  I opted for "notify me to schedule a restart," which seems sensible enough, right?

Wrong.  You see, when Windows Update "notifies" you, it does this by minimizing whatever you have open and popping up a system-wide modal dialog.

Yes, that's right.  A system-wide modal dialog (like a UAC prompt, so you can't click away from it).  Which steals focus and wasn't initiated by user action.

Seriously, guys?  That was the best UX design you could possibly come up with?

OK, let's review some basic UX rules that I thought Microsoft had down cold, but apparently they don't.  First, you do not steal focus.  Ever.  Whatever the user is doing is always more important than whatever you want to show them.  Microsoft applies this rule to other apps, but apparently can't be bothered to follow it themselves.  Second, dialogs are not modal.  To be more specific, dialogs should rarely be app-modal and (almost) never be system-modal.  UAC breaks this rule, but there are legitimate security reasons for it (if it were not system-modal, another app could steal focus or interact inappropriately with the dialog).  For Windows Update, there are no such reasons, other than the perennial "Windows Update must be as annoying as humanly possible" design aesthetic that Microsoft seems to go for with each new version of Windows.  Thirdly, dialogs are initiated by user actions.  Dialogs do not randomly appear when the user is in the middle of something, if at all avoidable.  For that use case, we have toasts, balloon tips, etc., which are all far less annoying.

It is 2015.  We should not be having this conversation, Microsoft.

Mercurial koans

A young acolyte approached Master Hg.  "Master Hg, what is the nature of Mercurial Branches?"

Master Hg replied, "Branches are markers attached to every commit, indelible and eternal."

The acolyte nodded.  "Master, if we use branches for tracking new features in our product, how are we to avoid namespace pollution?"

Master Hg regarded the acolyte coolly.  "Mu.  Branches are lines of development, forgettable and ephemeral."

At once, the acolyte was enlightened.

---

A student was working under Master Hg.  "Master, yesterday I discovered a seven-headed hydra in our history."

Master Hg nodded, saying nothing.

"I did not wish to fight the beast, but I noticed it resided entirely on a separate branch.  So I closed the branch, and thought it dealt with.  But today I looked again and saw the hydra still lived.  Why was the branch not closed?"

"The branch was indeed closed," replied Master Hg, "leaving six branches open."

---

Ed. note: This koan is obsolescent and has no successor.

One day, a traveler from a faraway land sought Master Hg's guidance.  "Master Hg, I wish to alter history."

Master Hg nodded, smiling warmly.  "What you seek is easily attainable.  History is supple and easily rewritten."

Excitedly, the traveler began researching in Master Hg's veritable library and shared his work with others.  After many days, he returned.  "Master, when I tried to share my changes with my friends, as I have done in my homeland, the DAG became extremely confused and I had to re-clone the server.  Why does Mercurial not work correctly?"

Anger flashed across Master Hg's face.  "What you seek is impossible.  History is unyielding and changing it the domain of the gods."

"But mere days ago," the traveler protested, "you told me otherwise."

"I find it curious you remember events which did not occur," replied Master Hg.

The traveler stormed out angrily.  It was many hours before enlightenment struck him.

Why I don't use hg-flow

I recently had a chance to read A successful Git branching model (the branching model underlying the popular git-flow and hg-flow extensions), and I found it rather interesting.  For NBTParse, I've been following a modified form of Mercurial's standard branching.  I thought about trying to adapt my work to use hg-flow, but I realized the differences are largely cosmetic:

• My main development descends from the @ bookmark, just like the "develop" branch of Driessen's model.  Conveniently, Mercurial automatically updates to this bookmark when cloning, if there's no obvious target revision.
• Although I rarely bother with them, feature branches are easily supported as bookmarked alternate heads of the default named branch.  I may use them more often once NBTParse approaches stability and it becomes necessary to keep the trunk stable(ish) leading up to a beta release.
• I use release branches, much like Driessen.  Mine are named branches instead of bookmarks, but this is mostly a matter of the former not existing under Git.  The release branches also have bookmarks, which are reused from one branch to the next; this makes it easy to (automatically) find the current unstable release branch (it's just release-unstable), for example.
• Much like feature branches, hotfix branches are just bookmarked alternate heads of release branches.  Again, I rarely bother with them, since my release branches remain open for as long as the released product is supported.  However, they can be useful if a fix is likely to require multiple commits or the attention of multiple developers.
• Now we come to the "master" branch.  I must admit, I don't quite have a master branch, but I have the next best thing.  All my releases are tagged, and the latest unstable (and stable, once we hit stable) is bookmarked.  I can just do hg log -r 'tag("re:version-.*")' to find everything that would have been on the master branch if I had one.  If I only want stable releases, I can use a more precise regex (e.g. ^version-\d+\.\d+\.\d+$).  Oh, and those revsets work in Bitbucket's search interface, too.

I'm sure Driessen's model is extremely helpful for him and lots of other developers, and it's nice to see people looking at nontraditional branching strategies.  But I guess I just don't see much value in rearranging my commits like this; my revision DAG is complicated enough already.

Watch Your Back, Git

Changeset evolution is a big deal.  But nobody seems to be talking about it.  Well, except for this guy:

But even he says it's a small set of incremental improvements.  This is not small.  But it is all a little abstract right now.  Let's write a use case.

The Pendulum and the Winch

I often find that explaining computer science to non-computer-scientists is difficult.  It's been said that computer science is like no other field of study.  Well, I think that's a rather strong claim to make.  What follows is a translation of a standard problem in computer science into physics.  It is an analogy, unrealistic but nevertheless interesting.

I have a pendulum, supported by some apparatus ultimately connected to a pillar or pole.  It is possible to move the apparatus up or down, but only by manually detaching and reattaching it by hand.  I have a winch affixed to this apparatus which may raise or lower the pendulum.  It is connected to a coil of rope or string (which, for the purposes of this problem, is infinitely long yet magically takes up a finite volume), and can be remotely controlled at the press of a button.  The winch is also geared discretely; it only turns in units, and then only one at a time.

I wish to carry out a series of experiments involving varying the length of my pendulum.  In particular, I often want to lengthen the pendulum.  Most of the time, this setup suits me quite well.  But sometimes, I find I need a pendulum longer than the apparatus is high off the ground.  In these situations, I need to climb the pillar and move the winch up.  In so doing, I may need to start an entire experiment over again because the pendulum lost energy while I was climbing.  How can I avoid or minimize those climbs in proportion to the maximum length of the pendulum?  We must assume I do not know the maximum length in advance, perhaps because my experiments are highly complex and difficult to predict, or perhaps because they are directed by someone else's instructions, and they did not think to tell me in advance how long a pendulum I would need.

Singularities happen all the time

We will soon create intelligences greater than our own. When this happens, human history will have reached a kind of singularity, an intellectual transition as impenetrable as the knotted space-time at the center of a black hole, and the world will pass far beyond our understanding. -- Vernor Vinge, 1983

 The technological singularity is supposed to occur when we develop "true" or "strong" AI.  Beyond that point, we are told, everything will be different, in the most conveniently vague ways.  Perhaps society will run on communism, or anarcho-capitalism, or something we don't have a name for (in other words, whatever the author happens to think would be ideal).  We are told that the resulting society will be totally incomprehensible to those of us still living in the present day.

My reaction to all this can be summed up in two words: "So what?"

Why I am not recommending GeoNode

For the past few weeks, I've been working for a professor on a project involving geographical data.  As part of this project, I was asked to evaluate GeoNode.  So I looked at the website, and after wandering around for a while trying to get past the usual marketing bullshit (side note: Can anyone explain to me why so many open source projects these days have such enormous quantities of marketing bullshit?), I eventually found some real documentation.  It was in the form of a developer "workshop," however, so I was a bit leery of it.

One-time pads with Python

A one-time pad is a kind of unbreakable encryption.  For most encryption, breaking it is a matter of throwing a lot of computational resources at the problem.  Typically, the amount of resources needed greatly exceeds the amount that is practical to obtain, so most modern cryptography is secure enough.  There are, however, some downsides to modern cryptography, the biggest of which is its complexity.  If we want to use crypto for something like bank transactions, complexity is not that big of a deal, because centralization can hide most of the complexity from end-users.  But if, for instance, you need to implement secure communications without a centralized certificate authority, effectively implementing secure communications becomes a lot harder.

An exercise

Sometimes, I feel like stretching my programming muscles.  A while ago, I read about this problem, or one like it.  It's not an especially hard problem, but I'd like to go over it anyway, because low-level data types can be unintuitive.

Suppose you have two integers a and b.  You want to add them, but you also want to be sure they don't overflow.  You're working in a language where overflow is undefined, such as C.  You can't use arbitrary precision variables or anything fancy like that, nor can you make use of low-level things like CPU overflow detection.  You need to guard against overflow mathematically, without the use of any of those systems.  Maybe you're working on a reduced architecture that doesn't provide overflow detection.  Maybe your language or library is deficient and lacks an arbitrary precision integer type.  It doesn't really matter.  The point is, taking away all those "outs", we're left with a somewhat interesting problem.

Here are some assumptions you may make:

1. You have a constant called INT_MAX which is equal to the largest integer which can be represented on your system.  You also have another called INT_MIN, which serves a similar purpose for the smallest (most negative) integer.
2. You're working in two's complement, but overflow is still undefined.
3. You do not have the exact number of bits available (but you could figure it out from INT_MAX, so that's not much of a restriction).
4. You may throw an exception to indicate an overflow condition.

Actually think about this problem for a few minutes.  I'll put the solution after the break.  I really think most people who are likely to read this blog can solve this on their own, so please actually try this.

Why I turned off Firefox's inline autocomplete

Firefox's location bar inline autocomplete is, for me, the single most annoying aspect of Firefox 14. Here's a list of why I'm annoyed:

1. Pressing tab navigates to the first dropped down suggestion. It has no relation whatsoever to the inline suggestion, contrary to appearances, and doesn't let me edit the URL before navigating to it. 
2. Pressing backspace removes the inline suggestion, but doesn't erase an actual character that I've typed. This throws off touch typing. 
3. When I'm typing a Google search, if Firefox thinks I made a typo, it adds ">>" followed by the "correct" search terms.  I've had false positives here, and pressing enter searches the whole thing, which I find confusing.  Maybe there's a "right" way to use this feature, but if that's the case, then in my opinion it's poorly afforded.  Or maybe Mozilla really thinks people want to search for both correct and incorrect terms.  I don't know.
4. Pressing enter goes to the autocompleted URL or search terms, instead of whatever I actually typed.  Since I often type rather quickly, this frequently results in searches for the wrong keywords.
5. If I want to edit the autocompleted URL (e.g. to add additional fragments to it), I have to press the right arrow.  This takes me off the home row.  It is intuitive to use tab for this purpose, but that does something entirely different (see point 1).
6. Last I checked, I wasn't able to find any addons on AMO to address any of these issues.  It's as if the new autocomplete fell out of the sky one day.  The only addons I was able to find were all about adding inline autocomplete to older versions of Firefox.

I like Firefox, and use it as my primary browser.   But I see this as a major blemish, and I went to the trouble of disabling it completely.  Here's how:

1. Go to about:config in the location bar (Blogger refuses to link to it, unfortunately).
2. If Firefox warns you, click through the warning.
3. Type browser.urlbar.autoFill in the Search box.
4. Double click the first result.
5. You're done!  No need to restart the browser or anything.

SpaceChem optimization

SpaceChem is a programming video game disguised as a chemistry video game.  A static description cannot really do it justice, so here's a trailer:

Honestly, the trailer doesn't do it justice either, so maybe you should go grab the demo from Steam.  Anyway, SpaceChem has two types of puzzles: Research and Pipeline puzzles.  Research puzzles are fairly straightforward to work with because an input/output operation always takes exactly one cycle, whereas with a Pipeline puzzle, the tubes pipes can be clogged or empty, making your I/O operations block.  This means that performance problems with one reactor can affect others.  But optimization is not an all-or-nothing affair.  You need to know where to concentrate your work.  That's what this post is about.

Do we even need copyright?

As we established last time, copyright is an indirect subsidy.  The purpose of this is to incentivize creativity, the idea being that art for its own sake is important, but won't pay the bills on its own.  But art has a nice property: people like it, especially if it's original and well-executed.  As there's both a supply and a demand for original, creative expression, I must wonder whether the market can connect the two without the use of a subsidy.

The notepad.exe problem

Quite a long time ago, there was a minor stir over notepad's treatment of text encodings. So many people talked about this that Raymond Chen even got in on the action (twice). As a Linux user, I honestly have no idea whether Microsoft ever got around to fixing this (Raymond's post suggests they consider it impossible to "fix" as such). I'm not really interested in bashing them for using an imperfect algorithm, since, to be honest, you'll never get perfection with this issue. But I do think it provides an interesting framework for a math problem.

Technically, ANSI is Windows-1252 (well, technically the term "ANSI" is wrong, but nobody cares), but in practice I think it's most commonly used as ASCII, in which case it's really equivalent to UTF-8. Now, I know that technically there are probably quite a lot of files out there with non-ASCII symbols encoded as 1252, but honestly, if you're trying to detect 1252 automatically, you've already lost since there are only 5 invalid code points; this means that if you feed totally random data into an ANSI decoder, it will take, on average, 50 bytes before the decoder complains of an invalid code point. This means that it's very likely to accept short runs of totally random data

One thing I notice is that, while Raymond says (implies, probably unintentionally) the string is ANSI, it's also perfectly good UTF-8, and will even produce the same interpretation in UTF-8 as in ANSI. This is, of course, because they're both backwards compatible with ASCII, which is what our string really is. Now, it's difficult to figure out the probability of random data being valid UTF-8 since UTF-8 is variable length. But I would say that it's rather unlikely since valid UTF-8 has quite a few constraints on its behavior: each leader byte must be followed by exactly the right number of continuation bytes, and overlong sequences are also disallowed. Then you look at Unicode itself and find that certain sequences (U+FFFF, for instance) are illegal.

UTF-16, on the other hand, is a much denser representation. Most 2 byte sequences are legal, unless they happen to form an invalid surrogate pair or result in an illegal code point (again, U+FFFF and friends). Now, the "illegal code point" problem is common to both encodings, so we can ignore it for our purposes (though technically, I suppose UTF-8 has the code points U+D800 to U+DFFF illegal, which UTF-16 can't even represent). But how likely is it that you get an invalid surrogate? Well, the likelihood of a random 16-bit value being (say) a leading surrogate is 1/64, since the first 6 bits are constrained (so you get 2^-6). This is the same for a trailing surrogate. Now we need the probability of a leading surrogate followed by something which is not a trailing surrogate, or a trailing surrogate followed by anything. Well, if we crunch the numbers, we find that it takes about 31 2-byte values, or 62 bytes, before you encounter an invalid surrogate pair in random data. This is actually worse than ANSI. However, that's not a valid comparison since we've been ignoring the invalid code points problem (which is different in ANSI). But there aren't a whole lot of "defined invalid" characters in the Unicode spec, so this only reduces the count a little (remember, the "reserved" U+D800 to U+DFFF cannot be safely represented in UTF-16; that's why they're reserved. Any attempt to do so will produce a (possibly illegal) surrogate pair, which we've already covered).

Earlier I said it's difficult to find the probability for UTF-8. I could, however, find the probability for a discrete code-point being invalid. This is less useful since it's not measured in bytes, but it will at least tell us how many characters your decoder will emit before it yells at you. So an invalid UTF-8 "character" is any of the following:

1. Begins with 0b10 (begins with a continuation byte, or the previous character has too many continuations). (p=1/4)
2. Begins with 0b110, but the next byte doesn't begin with 0b10. (p=3/32)
3. Begins with 0b1110, but either (or both!) of the next two bytes doesn't begin with 0b10. (p=15/256)
4. Begins with 0b11110, but any of the next 3 bytes isn't a continuation. (p=63/2048)
5. Begins with 0b11111 (too long). (p=1/32)
6. Begins with a valid leader byte, but the file ends before the requisite number of continuations have appeared. (p is impossible to calculate without knowing more about the file)

These possibilities are mutually exclusive (except for 6). We've ignored the issue of overlong encodings (other than 5; things beginning with 0xC0 and 0xC1) because they're difficult to account for (they unpleasantly overlap with 2)  and many decoders will silently allow them anyway, as they're unambiguous. If we assume the random data is long enough to avoid 6, we get P=91/2048. So your decoder will emit, on average, one character before it notices something illegal; this is equivalent to "no more than 4 bytes" since there's at most 4 bytes in a UTF-8 character. This is why UTF-8 is not supposed to need a BOM: it is very discriminating already, and it's highly unlikely for random data to be accepted as valid UTF-8. Just look at the PDF chart: after 3 characters (at most 12 bytes), random data has less than a 10% chance of being accepted. Of course, real data is seldom random, so applying this to IsTextUnicode is iffy at best. All this tells us is that, ceteris paribus, UTF-8 is much more likely to reject something which isn't UTF-8 than the other encodings.

So where does this leave us? Well, in my opinion, it leaves us with the conclusion that ANSI should be scrapped since it's both undiscriminating and BOM-less, making it rather difficult to autodetect; furthermore, UTF-8 provides ASCII-compatibility and a much more diverse character set. But Microsoft obviously can't do that since there are a lot of ANSI files floating around and people would be mildly annoyed if they had to be converted (seriously: because lots of legacy programs use ANSI and Microsoft hates breaking such programs, which cannot be easily converted (unlike text files); furthermore, I'm not a Windows programmer, but I don't think the ANSI interface is deprecated in the first place!).

I suppose it also suggests that Microsoft consider UTF-8 before other encodings, but as Raymond Chen said, "[N]o matter how you decide to resolve the ambiguity, somebody will win and somebody else will lose"; there's no general solution to this problem. And as we said before, real data may not follow the probability distributions that random data does.