Category Archives

4 Articles

Scrolling in div without scrolling page

The goal: Having a div on a page that scrolls with the mousewheel or a swipe, but that doesn’t scroll the page itself when the user reaches the bottom or the top. There’s a number of solutions out there – but I found them to variously work only on desktop or only on mobile, or only on certain browsers. The following Javascript/JQuery snippet, used for Chicmi, works on modern browsers and on mobile.

$(document).ready(function() {

    var touchPosY;
    var scrollableDiv = $('#the-target-div');

    scrollableDiv.bind('touchstart', function(event) {
        touchPosY = event.originalEvent.touches[0].clientY;
    });

    scrollableDiv.bind('touchmove', function(event) {
        var touchDelta = touchPosY - event.originalEvent.changedTouches[0].clientY;
        if (touchDelta > 0 && scrollableDiv.scrollTop() + scrollableDiv.height() == scrollableDiv.prop('scrollHeight')) {
            event.preventDefault();
        } else if (touchDelta < 0 && scrollableDiv.scrollTop() == 0) {
            event.preventDefault();
        }
    });

    scrollableDiv.bind('wheel', function(event) {
        if (event.originalEvent.deltaY > 0 && scrollableDiv.scrollTop() + scrollableDiv.height() == scrollableDiv.prop('scrollHeight')) {
            event.preventDefault();
        } else if (event.originalEvent.deltaY < 0 && scrollableDiv.scrollTop() == 0) {
            event.preventDefault();
        }
    });

});

Replicating PHP’s Adler32 hash in Python

If you’ve got to replicate a PHP hash('adler32', $string) command in Python, you may expect to just do this:

from zlib import adler32
checksum = adler32(string)

Unfortunately the values are different, for two reasons. Firstly, Python returns the result in decimal, while PHP returns the result in hex. Secondly, Python returns a signed value on some platforms and an unsigned value on others, so even within the same Python code you might not always get the same result.

Therefore, to get the same value on all platforms, and match the value you get back from PHP, you’ll need to use this:

from zlib import adler32
str(hex(adler32(email['email']) & 0xffffffff))[2:]

One in every 600 websites has .git exposed

One does not simply git pullFor web developers, exposing your .git folder to the world is a novice mistake. It allows anyone to download your entire source code repository, which often includes database passwords, salts, hashes, and third party API keys or usernames and passwords.

Over the years, for another personal project, I’ve built up a database of 1.5m reasonably respected domains. These are all either authority sites (such as the BBC or Guardian, or government, educational or military domains), or are domains that have inbound links from at least one of those types of sites.

Out of those 1.5m sites, 2,402 have their .git folder exposed and downloadable. That’s 1 in 600 decent respectable sites, or 0.16% of the internet, that is dangerously exposed.

Some of these .git repositories are harmless, but from a random sample many contain dangerous information that provides a direct vector to attack the site. Hundreds listed database passwords, or included API keys for services such as Amazon AWS or Google Cloud. Others included FTP details to their own web server. Many contained database backups in .SQL files, or the contents of hidden folders that are meant to be restricted.

One prominent human rights group exposed every single person who had signed up to a gay rights campaign (including their home address and email addresses) in a CSV file in their Git repository, publicly downloadable from their website. One company that sold digital reports provided its entire database of reports free of charge to anyone who wanted to download their .git folder.

So developers, please, please check that your .git folder is not visible on your website at http://www.yourdomain.com/.git/. If it is, lock it down immediately. Ideally delete the folder and find a better way to deploy your code, or at least make sure access is forbidden using an .htaccess. Then assume that someone has downloaded everything already and work out what they could have seen. What passwords, salts, hashes or API keys do you need to change? What data could they have accessed? What could they have done to alter or impair your service?

And then please spread the word among other developers too – because right now this must be one of the biggest holes in the internet.

Generating a simple geospatial index for MySQL

Searching for latitude and longitude values presents special problems for relational databases. By default databases like MySQL don’t handle queries like this very well:

SELECT * FROM table WHERE latitude BETWEEN w AND x AND longitude BETWEEN y AND z;

This is because it is not possible to index the latitude and longitude fields efficiently, so the query will almost always end up touching significantly more records than it needs to. It gets even worse if you try to implement a great circle lookup in SQL.

There are many ways to solve this problem – for example spatial extensions, or using a different database or search provider that supports spatial lookups.  However sometimes the requirement is so simple that you don’t want to overcomplicate your stack.

One answer is to provide your own spatial index to MySQL. A spatial index turns the world into a grid, and assigns each segment in the grid a unique identifier. There are many types of spatial grids – such as UTM or Marsden Squares. Often they were designed for a different era – many are alphanumeric, and many are complicated to calculate.

When approaching the problem at Chicmi, we decided that it was time for a simpler, quicker and numerical spatial index, that provided a numerical integer for every region of the world, and that could be used effectively in a relational database like MySQL. We came up with the PHP GridLatLong module.

With GridLatLong simply add one extra field wherever you have latitude and longitude fields – we call it gridref – which can be an INT. Make sure there’s an index on this field. Then when inserting or updating the latitude and longitude calculate the gridref value as follows:

$gridlatlong = new JamieMBrown\GridLatLong\GridLatLong();
$gridref = $gridlatlong->getGridReferences($latitude, $longitude)[0];

Insert that value in the gridref field alongside the latlong values.

Then when you want to search around a latlong, get all gridrefs within your desired search radius (in KM):

$gridlatlong = new JamieMBrown\GridLatLong\GridLatLong();
$gridrefs = $gridlatlong->getGridReferences($latitude, $longitude, $radius);

This returns an array of all of the matching gridrefs so that you can execute a query like this:

SELECT * FROM table WHERE gridref IN (gridrefs) AND latitude BETWEEN w AND x AND longitude BETWEEN y AND z;

This helps MySQL significantly narrow down the results to a much smaller area using its indexes, before doing the actual query.

There are a few extra options within GridLatLong – for example to make the grid more or less granular, or to change the units used for the search radius. Check out more over at the library page in GitHub. It’s freely available under an MIT license.