Let’s Encrypt on cPanel

Here is how I set up Let’s Encrypt for all sites on my hosted virtual private server running cPanel.

1. SSH as root

$ ssh -p 22 root@123.1.23.123

Your port might also be 2200. Ask your VPS hosting provider.

2. Then run the command:

$ /scripts/install_lets_encrypt_autossl_provider

3.Log into your main control panel:

https://123.1.23.123:2087

or however you access it (possibly port 2086 if http://)

4. Under SSL/TLS you’ll find “Manage AutoSSL”
Under “providers”, you’ll see “Let’s Encrypt”. That’s a new option that was created by running the command as root.

Select “Let’s Encrypt”. Then agree to their terms of service and create a new registration with Let’s Encrypt if necessary. Under the “managed users” tab you can enable / disable AutoSSL by account.

5. Now, under the control panel of each account, under SECURITY > SSL/TLS, under “Install and Manage SSL for your site (HTTPS)”, if you select “Manage SSL Sites”, you’ll see the Let’s Encrypt  cert.

Note: If you had a self signed certificate (which you don’t want), delete the cert in the individual account. Click the “run AutoSSL for all users” button as root under “Manage Auto SSL”. When you refresh the individual user, the correct cert should be there.

6. Yay. All your accounts now have an SSL Cert. You still need to redirect all of your http:// traffic to https://. In the .htaccess add the following:

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}/%{REQUEST_URI}

Now http://machinelearningworkshop.com/ redirects to https://machinelearningworkshop.com and http://www.machinelearningworkshop.com redirects to https://www.machinelearningworkshop.com/

and one of these days this blog will do the same.

W3C Performance Specifications

Here are some of the W3C’s web performance specifications:

  • High Resolution Time (Level 3)

    The DOMHighResTimeStamp type, performance.now method, and performance.timeOrigin attributes of the Performance interface resolve Date.now() issues with monotonically increasing time values with sub-millisecond resolution.

    https://w3c.github.io/hr-time

  • Performance Timeline (Level 2)

    Extends definition of the Performance interface, exposes PerformanceEntry in Web Workers and adds support for the PerformanceObserver interface.

    https://w3c.github.io/performance-timeline

  • Resource Timing (Level 3)

    Defines the PerformanceResourceTiming interface providing timing information related to resources in a document.

    https://w3c.github.io/resource-timing https://w3c.github.io/navigation-timing
    Supported in all browser except Safari and Opera Mini, starting with IE10

  • User Timing (Level 2)

    Extends Performance interface with PerformanceMark and PerformanceMeasure.

    https://w3c.github.io/user-timing
    Supported in all browser except Safari and Opera Mini, starting with IE10

  • Beacon API

    Defines a beacon API which can “guarantee” asynchronous and non-blocking delivery of data, while minimizing resource contention with other time-critical operations.

    https://w3c.github.io/beacon
    Not supported in IE, Safari or Opera Mini. Support started with Edge 14
    navigator.sendBeacon() on MDN

  • Preload

    Defines preload for resources which need to be fetched as early as possible, without being immediately processed and executed. Preloaded resources can be specified via declarative markup, the Link HTTP header, or scheduled with JS.

    https://w3c.github.io/preload

  • Cooperative Scheduling of Background Tasks

    Adds the requestIdleCallback method on the Window object, which enables the browser to schedule a callback when it would otherwise be idle, along with the associated cancelIdleCallback and timeRemaining methods.

    https://w3c.github.io/requestidlecallback

Capturing Captions from Youtube Videos

There are many tutorials on how to download the caption files created by Youtube if you own the video, but I was unable to find a way to download the captions of videos I don’t own. There’s probably an easy way to do it, but since I couldn’t find it, creating a JavaScript function to do it via the console took less effort.

Here’s the code (I did it three ways depending on how you like to code your JS)

Constructor method:

function CaptionCollector () {
  var that = this;
  this.captions = '';
  var nowShowing = '';

  this.collect = function(){
    try {
      var currentCaption = document.getElementsByClassName("captions-text")[0].innerText;
    } catch (e) {
      var currentCaption = null;
    }

    if(currentCaption && nowShowing != currentCaption) {
      nowShowing = currentCaption;
      that.captions += ' ' + nowShowing;
    }

    setTimeout(that.collect, 300);
  }
}

var foo = new CaptionCollector();
foo.collect();

Print the caption with foo.captions. Of course you can use anything instead of “foo”.

Here’s a version using JS object notation:

var captionCollector = {
    captions : '',
    nowShowing: '',

    collect : function(){
      try {
        var currentCaption = document.getElementsByClassName("captions-text")[0].innerText;
      } catch (e) {
        var currentCaption = null;
      }
    if(currentCaption && this.nowShowing != currentCaption) {
        this.nowShowing = currentCaption;
        captionCollector.captions += ' ' + captionCollector.nowShowing;
     }
    setTimeout(captionCollector.collect, 300);
  }
}

captionCollector.collect();

With this version, you print the console with captionCollector.captions

Or you can use the anonymous function method with a single global variable:

(function(){
    ___captions = '';
    var ___nowShowing = '';

    function getCaption() {
        try {
          var currentCaption = document.getElementsByClassName("captions-text")[0].innerText;
        } catch (e) {
          var currentCaption = null;
        }

        if(currentCaption && ___nowShowing != currentCaption) {
          ___nowShowing = currentCaption;
          ___captions += ' ' + ___nowShowing;
        }
        setTimeout(getCaption, 300);
    }

    getCaption();
})();

With this version, you print the console with the global variable ____captions

Because it uses the classname of the caption box Youtube uses for videos, this only works on Youtube. Alter the classname for other video services.

You do have to play the whole video to capture all the captions. With settings, you can play the video at twice the speed.

Clear the console when the video ends. Print the transcript to the console. Select all. Copy. You’re good to go.

TWELP: Twitter Help

Sometimes Twitter gets things wrong. Very, very, wrong. A few “features” that I think are bugs include Twitter Moments,

To this end, I created a little bookmarklet called “TWELP”.

<a href="javascript:(function(){function kill(){$('.promoted-tweet, .Icon--heartBadge').closest('li.stream-item').css('display','none');$('.js-moments-tab, .DismissibleModule').css('display','none');setTimeout(kill, 1000);}kill();})();">TWELP</a>.

Let me rewrite that for you in a version that’s easy to read, but won’t work to copy and paste:

(function(){
    function kill() {
         $('.promoted-tweet, .Icon--heartBadge').closest('li.stream-item').css('display','none');
         $('.js-moments-tab, .DismissibleModule').css('display','none');

         setTimeout(kill, 1000);
     }

     kill();
})();

The bookmarklet creates a kill function that:

  1. hides promoted tweets by finding the parent tweet containing a promoted-tweet child class
  2. hides any “liked” tweets that contains the heart icon, including uninteresting tweets in your stream suck as the fact that your friend Jane liked a tweet of a picture of her acquaintance Joe, who you are not following, eating an oyster. Seriously, who the fuck cares? It also hides the “people who liked your tweet” feature in your notifications. Not sure if that is a feature or a bug.
  3. hides the “Moments” tab by hiding the tab that has the  js-moments-tab class
  4. hides promoted modules that I hate like “In Case You Missed It” and “Who to follow”
  5. Calls itself once per second so if you scroll, it will continue killing those annoying tweets mentioned above.

TWELP – You can drag this link to your bookmarks bar, and click TWELP bookmarklet whenever you load Twitter. It kills the “Moments” tab, all ads, and removes the “X liked” tweets.

or, you can wrap your own.

Speed Perception

TL;DR: Please take the speedPerception challenge to help us confirm results about which web performance metrics best match human perception of speed.

Last summer, I was involved in a study called “SpeedPerception”, a large-scale web performance crowdsourced study focused on the perceived loading performance of above-the-fold content aimed at understanding what “slow” and “fast” mean to users. I am now involved in the second part of this study which aims to confirm (or refute) our findings.

SpeedPerception: the general idea

Traditional web performance metrics, like those defined in W3C Navigating Timing draft specification focus on timing each process along the content delivery pipeline, such as Time to First Byte (TTFB) and Page Load Time. SpeedPerception’s goal is to tackle the web performance measurement challenge by looking at it from a different angle: one which puts user experience into focus by focusing on the visual perception of the page load process. We show the user  sample video pairs of websites loading generated with http://www.webpagetest.org/ (WPT), and ask them which of the pair they perceive as having loaded faster.

In the first phase, we measured only Internet Retailer top-500 (IR500) sites in desktop size. Now we are testing whether the results we measured are true: in other words, do they only work for our IR500 sites on desktop? Will we get consistent results when testing  Alexa top-1000 (Alex1000) homepages? Will we see the same results if we test on mobile size screens with mobile lie-fi performance?

In this second phase, we’re testing both mobile and desktop versions of both IR500  and Alexa1000 website home pages. We’ve also added a way of measuring the user’s time to click so we can compare apples to apples.

The goal is to create a free, open-source, benchmark dataset to advance the systematic study of how human end-users perceive the webpage loading process: the above-the-fold rendering in particular. Our belief (and hope) is that such a benchmark can provide a quantitative basis to compare different algorithms and spur computer scientists to make progress on helping quantify perceived webpage performance.

Take the Speed Perception challenge !

How was SpeedPerception created?

Videos were created using  Patrick Meenan’s open-source WebPagetest (a.k.a WPT). We made 600+ videos of  2016 IR500 and Alexa-1000 home pages loading. The runs were done in February 2017. Videos were turned into gifs. Video pairs  were grouped using a specific set of rules to help limit bias and randomness. Everything is available on GitHub at https://github.com/pdey/SpeedPerception.

Open Source

Just like we did with the results of phase 1 of SpeedPerception, once the crowd-sourcing component generates a sufficient amount of user data, we will open source the dataset, making it available to the web performance community, along with the analysis of what we discover.

Please help us by taking the SpeedPerception Challenge now. Thanks.

Results from Phase 1

In phase 1, we discovered a combination of three values: an abbreviated SpeedIndex up to time to click (TTC) and an abbreviated Perceptual Speed Index up to TTC, in conjunction with startRender (or Render), can achieve upwards of 85%+ accuracy in explaining majority human A/B choices. Does the power of this new combination “model” hold true for all sites, or just our original data set? This is what we’re working on finding out.

If you’re interested in phase 1, here’s some more light reading:

 

Leak without a trace: Anonymous Whistle blowing

Instructions on how to leak data without getting caught

  1. Don’t leave digital traces while copying data.
  2. Write stuff down on a pad which belongs to you, and take it home.
  3. Photograph your screen. Don’t create files with copies of the data you are planning to exfiltrate on your work computer.
  4. If the data is on an internal web server, try to access what you are planning to leak in the form of multiple partial queries over a period of time, instead of as one big query.
  5. If you have large volumes of data, save it on a never-used-before USB drive and bring that home. Don’t ever use that USB drive for anything else again.
  6. Don’t use work-owned equipment to post from. Many employers have monitoring software installed, and will easily be able to see who posted what.
  7. Don’t use equipment where you’ve installed employer-supplied monitoring software to post.
  8. If you just read this, on equipment which might be monitored, wait a while before posting anything sensitive. Don’t give somebody who is watching a chance to correlate your seeing this with a post right after that.
  9. Install tor. You can get it at http://tor.eff.org. This is a special browser which is slow, but provides strong anonymity. It will prevent anybody at your ISP (if they’re watching) from knowing what sites you visit with it, and it will prevent any sites you visit from knowing what the IP address of your computer is. This will make it much harder for either of them to identify you.
  10. Use the Tor browser for posting your leak.
  11. You must follow these instructions to make sure tor really works: https://tor.eff.org/download/download-easy.html.en#warning.
  12. The instructions about never opening a downloaded file are vitally important — things like .doc and .pdf files can contain software which will expose who you are.
  13. The instruction about using an https version of a web site instead of the plain old http version is also very important. This is because while tor provides very strong anonymity, it doesn’t provide a secure connection to the web site — the https connection to the site does that.
  14. If you’re posting to a social site like twitter or reddit, or using an email account for it, you’ll need to set up a new account for posting your leak.
  15. If you’ve got an account which you have used when not on tor, then Google, twitter, reddit, etc. can identify the IP address of your computer from previous sessions.
  16. If you’ve ever posted anything on an account, there’s a good chance you’ve leaked information about yourself. Don’t take this risk, and just use a new account exclusively for leaking.
  17. Do NOT post files from standard editors, like Word, Excel, or photos from your camera. Most programs and recording equipment embed metadata in their files, like the identify of the creator, the serial number of a camera, and the likes in their files which can be used to identify you. Plain old .txt files are ok. Pretty much anything else risks your identity.
  18. When posting photos, be sure to use a metadata stripping tool like jhead for photos. Turn off photo syncing (e.g., iCloud).
  19. The secure drop systems used by some news outlets like ProPublica and the New York Times may be able to strip this kind of thing, but ask a journalist about your specific file format first.
  20. After posting the leak, you need to NEVER use that account for any purpose not directly tied to the leak, since you may make the mistake of giving away your identity.
  21. If you want to leak to the press (which may get broader coverage, but may also decide not to publish at all) organizations like ProPublica and the New York Times have special drop boxes set up to allow the posting.
  22. Details here:
    http://www.niemanlab.org/2017/01/how-easy-is-it-to-securely-leak-information-to-some-of-americas-top-news-organizations-this-easy/
    https://www.nytimes.com/newsgraphics/2016/news-tips/#securedrop

Web Performance Stats are Overrated

Note: A more polite version of this post originally appeared on Instart Logic’s blog

Who cares about performance?  Your customers, that’s who!  Speeding up your site by several seconds, or even by hundreds of milliseconds, will make your customers happy. Well, at least they’ll be less irritated.  That’s pure logic. I am not citing statistics. I am using common sense.

For example, a 50% decrease in file size — cutting bloat in half — has a greater impact on someone 2G mobile than on a T1 line. Common sense dictates the impact is greater on a 5Mb site than a 5Kb one, no matter the network speed as performance gains achieved by halving a 5Kb download may not even have a noticeable effect. Similarly, improving a 12s page load to 11s doesn’t have the same impact as improving a site from 3s down to 2s. The impact of a one second improvement depends on the original experience. This is obvious. It’s common sense.

Conclusions based on common sense are often accurate, but aren’t considered factual like conclusions based on statistics.  Statistics are often used to demonstrate something as fact, but a statistic is really only a fact about a piece of data. Mark Twain is credited with saying “Facts are stubborn things, but statistics are pliable.”

We discuss web performance statistics as if the web were a monolith, as if all web applications were uniform, like performance improvement effects are linear. They may not be represented by a simple exponential equation, but performance isn’t a simple science, and metric interpretation shouldn’t be assumed to be linear, or even accurate.

The statistic many web speeders quote “a one second delay in web page responsiveness leads to a 7% decrease in conversions, an 11% drop in pageviews, and a 16% decrease in customer satisfaction” is kind of bullshit today. This quote comes from a 2008 study: a study that came out a few months after the iPhone SDK was released and Android Market came into being, before either proliferated. When that study was conducted, the iPad and Android tablet were a few years off. As web speeders, we can’t quote a performance study that basically predates mobile. Yet, that’s what we’re doing.

While the customer satisfaction statistics might be BS, the conclusions are not. The longer the site takes to load, the less happy your customers will be, the more users will abandon your site, the lower your conversion rates, and the less money you’ll make. It’s not even that your customers will be unhappy — it’s that they may not be your customers anymore. That’s common sense, no matter what the actual percentages are.

Definitely give yourself a performance budget and test your site. Test your application throughout the development process. Optimize and right size your images, setting dimensions. Reduce DNS lookups. Reduce bloat, minimizing request size, GZipping all requests. Make fewer HTTP requests or serve content over HTTP/2, caching what you can. Basically, follow as many performance recommendations as you can.

Improve the performance of all of your content, not just your home page. Focus your energies on the critical path of your site. It’s obviously less important to optimize some pages, say the “libraries we use” page linked to from an “about us” page linked to from a political candidate’s website. But do realize users often enter your site thru a side door: for example, a product page, with the shopping cart experience being a necessity for all sales for an online store. In the case of eCommerce, optimizing your homepage might create the most “savings”, but will do little to improve user experience for your actual potential customers.

And never forget: improving the download speed is not enough: improving your site by one second delay won’t improve customer satisfaction if, once loaded, they’re met with an unresponsive UI.

Or maybe it will.

Let me go back to 2008 to test that out.

animation-iteration-delay hack

While there is no such property as an `animation-iteration-delay`, you can employ the `animation-delay` property, incorporate delays within your keyframe declaration, or use JavaScript to fake it. The best method for ‘faking it’ depends on the number of iterations, performance, and whether the delays are all equal in length.

What is an animation iteration delay? Sometimes you want an animation to occur multiple times, but want to wait a specific amount of time between each iteration.

Let’s say you want your element to grow 3 times, but want to wait 4 seconds between each 1s iteration. You can include the delay within your keyframe definition, and iterate through it 3 times:

.animate3times {
    background-color: red;
    animation: yellow;
    animation-iteration-count: 3;
    animation-duration: 5s;
 }
@keyframes yellow {
 80% {
    transform: scale(1);
    background-color:red;
  }
  80.1% {
    background-color: green; 
    transform: scale(0.5);
  }
  100% {
    background-color: yellow;
    transform: scale(1.5);
  }
 }

Note the first keyframe selector is at the 80% mark, and matches the default state. This will animate your element 3 times, staying in the default state for 80% of the 5 second animation, or for 4 seconds, and moving from green to yellow and small to big over the last 1 second of the animation duration, before iterating again, stopping after 3 iterations.

This method works for infinite repetitions of the animation as well. Unfortunately, it is only a good solution if the iteration delay between each iteration is identical. If you want to change the delay between each iteration, while not changing the duration of the change in size and color, you have to write a new @keyframes definition.

To enable different iteration delays between animations, we could create a single animation and bake in the effect of three different delays:

.animate3times {
    background-color: red;
    animation: yellow;
    animation-iteration-count: 1;
    animation-duration: 15s;
 }
@keyframes yellow {
  0%, 13.32%, 20.01%, 40%, 46.67%, 93.32% {
    transform: scale(1);
    background-color:red;
  }
  13.33%, 40.01%, 93.33% {
    background-color: green; 
    transform: scale(0.5);
  }
  20%, 46.66%, 100% {
    background-color: yellow;
    transform: scale(1.5);
  }
 }

This method may be more difficult to code and maintain. It works for a single cycle of the animation. To change the number of animations or the iteration delay durations, another keyframe declaration would be required.

The animation-iteration-delay hack

There’s a solution that currently works that is not specifically allowed in the animation specification, but it isn’t disallowed, and it’s supported: you can declare an animation multiple times, each with a different animation-delay.

.animate3times {
    animation: yellow, yellow, yellow;
    animation-delay: 0, 4s, 10s;
    animation-duration: 1s;
 }
@keyframes yellow {
  0% {
    background-color: green; 
    transform: scale(0.5);
  }
  100% {
    background-color: yellow;
    transform: scale(1.5);
  }
 }

See http://codepen.io/estelle/pen/PqRwGj.

We’ve attached the animation three times, each with a different delay. In this case, each animation iteration concludes before the next one proceeds. If they overlap, while they’re concurrently animating, the values will be the values from the last declared animation.

See http://codepen.io/estelle/pen/gpemWW

Of course, you can also use JavaScript with animationstart, animationiteration and animationend eventlisteners that add or remove animation names or classes from your element.

Survey for US Developers’ Experience

I created a survey of developer experiences for US based developers.
While the results aren’t scientific in the the questions and optional answers were not all that well written, and the distribution may have been biases (only reached Twitter users), some of the results were indeed interesting, and may lead to me doing some further, more professional, study.

According to their own self reporting, men experience the following more than women do:

  • People assume more men developers have a CS degree (1.18)
  • People think men developers are very skilled at tech when they first meet them. (0.89)
  • People assume men  developers are good at math (0.80)
  • People make accurate judgements as to men developers ‘s skill set when they meet them (0.73)
  • Men developers sign up for conferences or RSVP for meetups and let their bosses and partners know the plans them made (0.65)
  • People assume men developers are gamers (0.59)
  • People listen to men developers  (0.54)
  • People assume male developers are straight (0.52)
  • Men developers  provide constructive criticism to others online (0.51)
  • People value men developers ‘s opinion (0.48)
  • People assume men  developers are correct (0.45)

According to their own self reporting, women experience the following more than men do:

  • People assume women developers are not technical when they meet them (2.22)
  • Women developers  are  told to be “nice” or to “smile” (1.98)
  • Women developers  have been called bossy (1.80)
  • Women developers  are the ones that makes sure they have a housesitter, babysitter, dogsitter, other so they can attend events such as conferences. (1.75)
  • Women developers have been called aggressive 1.65
  • People question women developers ‘s accuracy and/or knowledge (1.42)
  • Women  developers think about transportation to and from the venue before agreeing to attend events (1.25)
  • People explain women  developers ‘s own content to them, like their tweets and posts (1.18)
  • Women  developers go into work on the weekend (1.18)
  • People assume women  developers  know photoshop or have other design skills (1.14)
  • People comment on women developers ‘s appearance 1.12
  • Women  developers think about the different ages of their co-workers (1.06)
  • People interrupt women developers (1.00)
  • People are surprised to learn women developers are gamers  (0.98)
  • Women  developers think about what they are going to wear to work. (0.98)
  • People think women developers are administrative or custodial staff (0.95)