jamiembrown


Scrolling in div without scrolling page

The goal: Having a div on a page that scrolls with the mousewheel or a swipe, but that doesn’t scroll the page itself when the user reaches the bottom or the top. There’s a number of solutions out there – but I found them to variously work only on desktop or only on mobile, or only on certain browsers. The following Javascript/JQuery snippet, used for Chicmi, works on modern browsers and on mobile.

$(document).ready(function() {

    var touchPosY;
    var scrollableDiv = $('#the-target-div');

    scrollableDiv.bind('touchstart', function(event) {
        touchPosY = event.originalEvent.touches[0].clientY;
    });

    scrollableDiv.bind('touchmove', function(event) {
        var touchDelta = touchPosY - event.originalEvent.changedTouches[0].clientY;
        if (touchDelta > 0 && scrollableDiv.scrollTop() + scrollableDiv.height() == scrollableDiv.prop('scrollHeight')) {
            event.preventDefault();
        } else if (touchDelta < 0 && scrollableDiv.scrollTop() == 0) {
            event.preventDefault();
        }
    });

    scrollableDiv.bind('wheel', function(event) {
        if (event.originalEvent.deltaY > 0 && scrollableDiv.scrollTop() + scrollableDiv.height() == scrollableDiv.prop('scrollHeight')) {
            event.preventDefault();
        } else if (event.originalEvent.deltaY < 0 && scrollableDiv.scrollTop() == 0) {
            event.preventDefault();
        }
    });

});

Replicating PHP’s Adler32 hash in Python

If you’ve got to replicate a PHP hash('adler32', $string) command in Python, you may expect to just do this:

from zlib import adler32
checksum = adler32(string)

Unfortunately the values are different, for two reasons. Firstly, Python returns the result in decimal, while PHP returns the result in hex. Secondly, Python returns a signed value on some platforms and an unsigned value on others, so even within the same Python code you might not always get the same result.

Therefore, to get the same value on all platforms, and match the value you get back from PHP, you’ll need to use this:

from zlib import adler32
str(hex(adler32(email['email']) & 0xffffffff))[2:]

Educational fashion events in London

This is a WordPress demo of Chicmi’s embedded fashion calendar for bloggers, and shows educational fashion events happening in London during the next seven days.

It could also show London sample sales.

One in every 600 websites has .git exposed

One does not simply git pullFor web developers, exposing your .git folder to the world is a novice mistake. It allows anyone to download your entire source code repository, which often includes database passwords, salts, hashes, and third party API keys or usernames and passwords.

Over the years, for another personal project, I’ve built up a database of 1.5m reasonably respected domains. These are all either authority sites (such as the BBC or Guardian, or government, educational or military domains), or are domains that have inbound links from at least one of those types of sites.

Out of those 1.5m sites, 2,402 have their .git folder exposed and downloadable. That’s 1 in 600 decent respectable sites, or 0.16% of the internet, that is dangerously exposed.

Some of these .git repositories are harmless, but from a random sample many contain dangerous information that provides a direct vector to attack the site. Hundreds listed database passwords, or included API keys for services such as Amazon AWS or Google Cloud. Others included FTP details to their own web server. Many contained database backups in .SQL files, or the contents of hidden folders that are meant to be restricted.

One prominent human rights group exposed every single person who had signed up to a gay rights campaign (including their home address and email addresses) in a CSV file in their Git repository, publicly downloadable from their website. One company that sold digital reports provided its entire database of reports free of charge to anyone who wanted to download their .git folder.

So developers, please, please check that your .git folder is not visible on your website at http://www.yourdomain.com/.git/. If it is, lock it down immediately. Ideally delete the folder and find a better way to deploy your code, or at least make sure access is forbidden using an .htaccess. Then assume that someone has downloaded everything already and work out what they could have seen. What passwords, salts, hashes or API keys do you need to change? What data could they have accessed? What could they have done to alter or impair your service?

And then please spread the word among other developers too – because right now this must be one of the biggest holes in the internet.

Generating a simple geospatial index for MySQL

Searching for latitude and longitude values presents special problems for relational databases. By default databases like MySQL don’t handle queries like this very well:

SELECT * FROM table WHERE latitude BETWEEN w AND x AND longitude BETWEEN y AND z;

This is because it is not possible to index the latitude and longitude fields efficiently, so the query will almost always end up touching significantly more records than it needs to. It gets even worse if you try to implement a great circle lookup in SQL.

There are many ways to solve this problem – for example spatial extensions, or using a different database or search provider that supports spatial lookups.  However sometimes the requirement is so simple that you don’t want to overcomplicate your stack.

One answer is to provide your own spatial index to MySQL. A spatial index turns the world into a grid, and assigns each segment in the grid a unique identifier. There are many types of spatial grids – such as UTM or Marsden Squares. Often they were designed for a different era – many are alphanumeric, and many are complicated to calculate.

When approaching the problem at Chicmi, we decided that it was time for a simpler, quicker and numerical spatial index, that provided a numerical integer for every region of the world, and that could be used effectively in a relational database like MySQL. We came up with the PHP GridLatLong module.

With GridLatLong simply add one extra field wherever you have latitude and longitude fields – we call it gridref – which can be an INT. Make sure there’s an index on this field. Then when inserting or updating the latitude and longitude calculate the gridref value as follows:

$gridlatlong = new JamieMBrown\GridLatLong\GridLatLong();
$gridref = $gridlatlong->getGridReferences($latitude, $longitude)[0];

Insert that value in the gridref field alongside the latlong values.

Then when you want to search around a latlong, get all gridrefs within your desired search radius (in KM):

$gridlatlong = new JamieMBrown\GridLatLong\GridLatLong();
$gridrefs = $gridlatlong->getGridReferences($latitude, $longitude, $radius);

This returns an array of all of the matching gridrefs so that you can execute a query like this:

SELECT * FROM table WHERE gridref IN (gridrefs) AND latitude BETWEEN w AND x AND longitude BETWEEN y AND z;

This helps MySQL significantly narrow down the results to a much smaller area using its indexes, before doing the actual query.

There are a few extra options within GridLatLong – for example to make the grid more or less granular, or to change the units used for the search radius. Check out more over at the library page in GitHub. It’s freely available under an MIT license.

Forget Jurassic World, Ex Machina is the spiritual sequel to Jurassic Park

Jurassic World – the fourth Jurassic Park movie – launches on June the 10th, almost 14 years after JPIII launched in 2001. And as the marketing machine winds up and prepares us to return to Isla Nublar, you can be sure that there will be no escape from velociraptors, motorbike chases, greedy execs, kids in peril, and the inevitably catastrophic new superdino.

The marketing message is clear – this film will be bigger, better, louder, larger and more dangerous than anything that’s come before. It’s the newer, even higher, even faster rollercoaster of the dinosaur movies.  And in all that noise, when looking back on Jurassic Park (1993!) you’d be forgiven for just remembering the parts that Jurassic World is so keen to exceed – the ground-breaking special effects, the stupendous tyrannosaur, the terrifying call of the velociraptor, and *that* impact tremor scene.

But it’s easy to forget that Jurassic Park wasn’t really a story about dinosaurs. In the entire 127 minutes of film, only 14 minutes contained dinosaur effects shots.  They may be the scenes that left us with an impact, and that made us want to go back to the islands again and again, but it’s the movie’s slow-paced philosophical crescendo that makes it one of the great sci-fi movies of its time. It has not only had an impact on film technology, but also in attitudes towards science and technology in popular culture.

And not least in 2015’s Ex Machina. On the surface, Ex Machina is a completely different type of movie to Jurassic Park. It’s about AI and not dinosaurs, it lacks any of Jurassic Park’s family-friendly trimmings, and its deep-dive into its scientific and philosophical themes makes Jurassic Park look like the scientific-equivalent of a Kraft Dinner to Ex Machina’s rib-eye steak. But Ex Machina owes so much to JP – both in the way the story is told, and the use of science to drive the drama.

Let’s start with the protagonists. In both films they are technical experts – in JP a palaeontologist and a palaeobotanist, in Ex Machina a coder. They have been asked to go somewhere, but they don’t know why and they don’t know what to expect when they get there.

Jurassic Park ProtagonistsEx Machina ProtagonistCue a helicopter ride over spectacular scenery to a mysterious remote location.

Photo 22-05-2015 08 17 25Helicopter in Ex MachinaThe protagonists are accompanied eccentric billionaire, who has done something groundbreaking. We know what it is, but our heroes still do not.

Photo 26-05-2015 00 43 29Caleb meets NathanFirst we establish the rules of the location. Both plots rely heavily on computer-controlled electronic security systems.

Photo 26-05-2015 00 44 20Keycard system in Ex MachinaThen we meet the star attraction. Both the protagonist and the audience are entranced. Even though we knew what was coming, we’re amazed as we see it through their eyes.

Alan and Ellie Meet the DinosaursCaleb meets AvaThere are signs of danger:

Ellie discovers the T-RexCaleb and the Cracked GlassBut first, here comes the science!

Jurassic Park DNA CartoonEx Machina talks science!And next the philosophical debates. Remember that Jurassic Park spends about 10 minutes around a dinner table with characters talking about science, genetics and morality.

Photo 22-05-2015 08 20 22Ex Machina PhilosophyBoth films use a thought experiment to share scientific concepts – in Jurassic Park Malcolm uses one to discuss chaos theory, in Ex Machina, Caleb uses one to discuss the human mind vs the computer mind.

Malcolm demonstrates Chaos TheoryEx Machina Mary and the Black and White RoomBoth movies take trips off into related topics that aren’t core to the action. Which Hollywood movie can claim to spend 5 minutes investigating a sick animal by searching through piles of dung?

One Big Pile of ShitWhile Ex Machina gives us a lesson in art history.

Ex Machina and Jackson PollockBoth have a respect for – and fear of – computer technology, with extensive shots of code:

Samuel L Jackson hackingPython in Ex MachinaBut ultimately the topic of the film wants to get out!

Raptors test the fencesAI wants out!Someone messes with the security systems:

The fences are offCaleb messes with the security… and life, ah, finds a way:

Grant vs the T-RexEx Machina OopsBoth films also deal heavily in gender politics. But while Ex Machina may have a fembot problem and gets a resounding F in the Bechdel test, Jurassic Park passes (by a whisker), and as Ellie would say:

Ellie and sexism in survival situations