How to Lie with Statistics – Darrell Huff

20200623 How to Lie with Statistics

I bought this book many years ago when I was employed by the accounts department of a large UK firm to analyse the figures and produce reports for the board of directors on performance of all aspects of the business not just financial. Now you may think that purchasing a book entitled How to Lie with Statistics would suggest that these board reports may not have been entirely accurate; but in fact I got it for the same reason as it was written because if you know how things can be done badly then you can avoid making the same ‘mistakes’. Unless of course you are trying to show something, or more likely hide something, in the numbers, in which case the book becomes even more useful as a source of helpful hints. Rereading it at a time when we are bombarded with statistics and graphs (oh how a lover of selective data loves graphs) relating to the global pandemic of Covid-19 adds a useful dose of cynicism which we could all do with and the cartoons by Mel Calman are as pointed as they so often are.

Averages and relationships and trends and graphs are not always what they seem. There may be more in them than meets the eye and there may be a good deal less.

The book is full of examples of misleading statistics either real ones or created data to illustrate a point, for example just what is an average? Now the lay person reading that the average of something is say five will assume that tells you something, but which definition of average is being used? There are after all three main types all of which can give wildly different results depending on what you want to prove. The mean is what most people assume is an average that is add up all the numbers and then divide by how many numbers are in the sample. But then there is the median which is simply the middle number if you write out the data in numeric order, now this is useful for getting rid of weird data in the sample, the series 1, 3, 3, 5, 7, 9, 147 has a median of 5 which is ‘probably’ more useful than the mean of that data set which would push the ‘average’ much higher than all but one of the numbers in the set but it can also be misleading if that answer of 147 turns out to be important and you have simply ignored it. The only other average most people will come across is the mode, now that is simply the number that occurs most often so in the previous example that would be 3. So is the average 3, 5 or 25? Well it depends what you want to prove all of them are legitimate averages. In the book Huff uses a similar example where the data is household income, if my sample is also monthly income in thousands of pounds then all we have proved is that this particular group probably includes a professional footballer on £147,000 a month. Saying that the average is £25,000 a month is meaningless unless you want to imply that this is a particularly wealthy neighbourhood to property investors that haven’t been there but under one definition it is the average income, so should they build a Waitrose or an Aldi supermarket?

Each chapter features different ways of presenting data starting with samples with built in bias. A postal survey asking if people like filling in postal surveys may well show that 95% do, but unless you also know that they sent out 100,000 surveys and only got 250 back you don’t see the 99.75 percent of people polled that so dislike filling in postal surveys they simply threw it away. A famous real example of this mentioned in the book is The Kinsey Report on the sex lives of Americans in the 1940’s and early 1950’s. This report claimed to be revolutionary and is still cited but how many people back then were going to be willing to take part in the survey? By the nature of the responding sample we have another self selecting group biased towards people who are more open about their sex lives and preferences and may also on that basis be more experimental therefore skewing the results.

But to really lie with statistics you need a graph which is why politicians and marketing departments love them so much, one of the examples in the book is reproduced below and shows a oft repeated trick to make figures look more impressive, truncating the vertical axis, both graphs show the same data but have a different title to reflect what the story is.

20200623 How to Lie with Statistics 2

Another popular trick with graphs is to start or stop the range displayed to avoid including inconvenient data, if a graph based on monthly figures doesn’t start in January or maybe starts in 2007 (which seems an odd year to choose unless mapping something that did actually commence then) always ask the question what were the figures that preceded those displayed, likewise if it appears to stop at a random point then that is probably where the data stopped matching whatever the person drawing the graph wanted to prove.

Percentages are also to be looked at carefully, percentage of what precisely is always a good question. If something is £10 now and £15 next year it is 50% more expensive but the reverse isn’t the case, something £15 and £10 next year is 33% cheaper however it’s amazing how often you see the figure of 50% being used, an example is of the president of a flower growers association in the US who claimed flowers are 100% cheaper than they were last year, what he meant was that the price last year was 100% higher than now, if they were really 100% cheaper they would have to give them away. There are lots more examples in the book and you don’t need any mathematical knowledge to understand any of them, Huff is really good at explaining just why you should be always looking twice at any statistic and the more simplistic the way it is presented then the more cynical you should be.

Darrell Huff wrote this classic back in 1954 and it was then published by Victor Gollancz and first editions now sell for many hundreds of pounds. This is the 1973 first Pelican Books edition and it was Pelican that commissioned Calman’s drawings and is much more reasonably priced. It doesn’t appear to still be in print but copies are easy to find on the secondhand market. Now more than ever this book is needed.

20200623 How to Lie with Statistics 3

 

Mathematics in the Time of the Pharaohs – Richard J Gillings

20200421 Egyptian mathematics 1

There are four extant sources for this book, the Egyptian Mathematical Leather Roll (EMLR), the Reisner Papyri (RP), the Moscow Mathematical Papyrus (MMP) and the most important The Rhind Mathematical Papyrus (RMP) which was actually a training manual for scribes. Because it is there to teach this final papyrus document is crucial to our understanding of how the ancient Egyptians performed their calculations. This document along with the EMLR are in the British Museum in London, the RP is in the Boston Museum of Fine Arts whilst the MMP is in the Pushkin Museum of Fine Arts in Moscow, as you would expect from its name. Gillings wrote his book in 1971 and one or two errors have since been noted in the mathematical press as further studies have been made of the four sources and almost fifty years have passed since he wrote it, but these are largely technical and the book is mainly correct especially in its overview as to how ancient Egyptians calculated and is pretty comprehensive. Having said that it is definitely not a book for the layman, it is pretty solid mathematics and I would suggest that there is still a gap in the market for a simpler presentation which would introduce those with a curiosity in the subject to more easily come to some understanding as to how this worked.

Ancient Egyptian mathematics was largely overlooked and dismissed by scholars as simplistic especially when compared to that of ancient Greece but that overlooked the fact that it was more than capable of calculating the dimensions of the pyramids. For instance if you want a pyramid 139m high (the size of the Great Pyramid at Giza) just how big a base do you need to start with? It has also ensured that they still haven’t fallen down thousands of years after construction. Also the Egyptians had a large empire, so built irrigation canals, large granaries and temples, and of course had a comprehensive tax system to pay for all this so they must have been highly capable at least in the field of applied mathematics and engineering.

But let’s start at the beginning with the hieroglyphic representation of numbers, one is simply a vertical line, two is two of these and so on until you have nine lines drawn together to represent nine. Now this is easy and like all tally marks rapidly becomes unwieldy so you need symbols to indicate larger numbers and the earlier forms of these are shown below. I love the symbol for a million which appears to be the scribe throwing his hands in the air as if to say wow what a big number, what do I do with this?

20200421 Egyptian mathematics 2

In fact the papyrus scrolls were written in hieratic script which is sort of cursive hieroglyphic and is much more difficult to read and it is also important to note that they wrote right to left just as in modern Arabic so to our way of looking at it you would see the units first, then the tens, hundreds etc. There is a quick way of remembering this as animals or birds used in hieroglyphic writing always look towards the direction the writer. In the scrolls we have available to modern study addition and subtraction are regarded as so simple as to not need to show any working out which is unfortunate as this means we don’t actually know how they did it, you just get the required sum and then the answer. However everything beyond that is included and it should be understood that the ancient Egyptians managed their entire means of calculation by merely being able to multiply and divide by two and for reasons that are too complex to go into in this blog they also had the 2/3 times table (usually written down rather than memorised) and used this so extensively that when they needed to find a third of a number they would first get two thirds of it and then halve the answer.

So how did they multiply? Well the example given in the book is for multiplying 7 by 13 and this was done as follows. Start by writing two columns, the first of which has a 1 in it and the second has one of the numbers to be multiplied (this is the second example in the book as I think it is better to understand than the first). Under each number double the figure above until doing so in the column starting with one you would have a number larger than the number you are trying to multiply.

20200421 Egyptian mathematics 3

Simply adding up the values opposite the checked values in the first column gives the answer to 7 x 13 which is 91. If the number in the first column isn’t needed to sum to 13 in this case then you simply ignore the corresponding number in the second column. It’s simple really. Division is done the same way but a scribe asked to divide 184 by 8 would instead ask himself how many times do I need to multiply 8 to get to 184 so would create a similar chart to the one above.

20200421 Egyptian mathematics 4

Now at this point you hit the issue of fractions which we know that they understood as they had the 2/3 table but the way the ancient Egyptians handled them is definitely beyond me being able to explain here, I will simply say that with the sole exception of 2/3 they did not have any fractions with numerator other than 1, so to express ¾ for example they would write the equivalent of ½ + ¼. As you can imagine this becomes extremely messy very quickly. But the way they expressed a fraction, especially in hieratic is interesting as they drew a line over the number to indicate that it was a fraction and as the numerator was always 1 they didn’t need to show this, two thirds had it’s own specific character so that didn’t cause confusion. Later mathematical systems simply added a numerator above the line to indicate multiples of the denominator so this is where our way of writing fractions almost certainly originates from.

I only want to include one further example from the book and this one I chose as I particularly liked the calculation. I do recommend seeking out this book or the various online papers now available on the subject if you want to take this interesting branch of mathematics further. The calculation is how did they work out the area of a circle? Now courtesy of the ancient Greeks and their discovery of geometry (Euclid in particular) we know that the area of a circle is the square of the radius multiplied by the irrational number π which is 3.14 to two decimal places and that will do for most calculations. Archimedes worked it out to about that in 250BC but that is over 1300 years later than our poor scribe in ancient Egypt so how did he do it?

Well their calculation as given in problem 50 of the RMP is to take the diameter, work out a ninth of that figure, subtract that from the original number and then square the result. The sample problem takes a circle with a diameter of 9 khet (which makes the maths a lot easier), so if the diameter is 9 then a ninth of that is 1, subtracting that from the diameter gives 8. Multiply 8 by 8 as we know how to do above and that gives 64 setat as the area of the circle. It’s a hell of a big circle though as a khet is about 57 yards (roughly 52 metres) so a single setat, or square khet, is roughly 3250 yd² or 2720 m². But how accurate is the result of 64? Well (4½)² x π, which is our way of doing the calculation, gives 63.62 so it’s pretty good. If I ever needed to work out the area of a circle in my head (oddly something I don’t do often) then the ancient Egyptian way is definitely the way to do it, no messing around with π needed.

I was intrigued, so some basic algebra (also not something I do every day anymore) shows that the way the calculation works means that their equivalent of π is 256 / 81 which is 3.16 to two decimal places which explains the accuracy. It is also by far the simplest calculation that is remotely accurate as simply subtracting a ninth from the diameter is very easy. I spent a little more time with a calculator and found that the next easiest fraction that gives a better result is to subtract 4/35ths  which is lot more difficult than dividing by 9.

There is considerably more in the book for a keen mathematician to have fun with such as calculations of volumes and of course the all so important dimensions of a pyramid and truncated pyramid (i.e. one still under construction). So once you can get past the slightly confusing way it is written it’s fun to work through, preferably with some paper and a pencil nearby to do some quick calculations of your own.

Flatland – Edwin A. Abbott

20190416 Flatland 1

I asked my friend, Catalan booktuber Anna, best known under her nom de plume of Mixa, to choose this weeks read from a random group of titles I provided and she selected Flatland because she had never heard of it and was intrigued by the idea of a mathematical classic combined with social parody. Written in 1884 by an English headmaster who specialised in ‘classics’ i.e. Greek and Latin; this is as an unlikely cornerstone of multi-dimensional non-Euclidean geometry as can really be imagined. I first read it in my teens and although the copy on my shelves is from my mid twenties I probably haven’t read it in over two decades so it is well worth revisiting.

The book is split into two sections, the first describes Flatland and it’s inhabitants whilst the second deals with one of it’s inhabitants A. Square and his perspective of several other lands. Initially Lineland, then what is called Spaceland which is our own set of dimensions and finally Pointland before he finally returns to his own two dimensional world and the prison that we find him in at the start of the narrative.

But let us begin with a description of Flatland because it is with an understanding of this two dimensional land that we will start to see the effects of an extra dimension which is not apparent to the inhabitants. Our narrator A. Square is as you might expect a square and as such is a lawyer, the number of sides that each character has denotes his status in society as follows:

Our Middle Class consists of Equilateral or Equal-Sided Triangles. Our Professional Men and Gentlemen are Squares (to which class I myself belong) and Five-Sided
Figures or Pentagons. Next above these come the Nobility, of whom there are several degrees, beginning at Six-Sided Figures, or Hexagons, and from thence rising in the number of their sides till they receive the honourable title of Polygonal, or many-sided. Finally when the number of the sides becomes so numerous, and the sides themselves so small, that the figure cannot be distinguished from a circle, he is included in the Circular or Priestly order; and this is the highest class of all.

It is a Law of Nature with us that a male child shall have one more side than his father, so that each generation shall rise (as a rule) one step in the scale of development and nobility. Thus the son of a Square is a Pentagon; the son of a Pentagon, a Hexagon; and so on.

Below the Equilateral triangles are the ranks of workers and soldiers who are Isosceles and as the size of the smallest angle contained within a figure is an indication of intelligence clearly the more ‘pointed’ such a triangle is the lower the intellect and (bearing in mind this is a Victorian book) the more violent and criminal the individual is assumed to be. Rather than increasing sides with each generation Isosceles triangles gain half a degree to their smallest angle each time until they are finally assessed to be Equilateral and the family can then start to rise through society.

Now it should be noted that as indicated in the quote above this only applies to sons, so what about the females, well they are all just straight lines and this is where Edwin Abbott Abbott (yes the A. in his name really was Abbott as well) hit accusations of misogyny even in the 1880’s. Something he attempted to address in a preface added to the second and revised edition but without much success, one of the more offending sections being below…

Not that it must be for a moment supposed that our Women are destitute of affection. But unfortunately the passion of the moment predominates, in the Frail Sex, over every other consideration. This is, of course, a necessity arising from their unfortunate conformation. For as they have no pretensions to an angle, being inferior in this respect to the very lowest of the Isosceles, they are consequently wholly devoid of brain-power, and have neither reflection, judgement nor forethought, and hardly any memory.

Still enough of the first half of the book, there are lots of details given as to how houses are constructed, how the people recognise each other and various social mores which whilst interesting in the way Abbott has tried to give life to his creation do not really impinge on the main object of the book which is contained in part two. The important section is in the remainder where A Square visits other lands and learns about dimensions other than the North/South, East/West directions he is currently familiar with. The first of these is described as a dream where he perceives Lineland a place of just one dimension with all the inhabitants travelling over a single line with him floating over it so that he can see along the line.

 

20190416 Flatland 5

As A. Square interacts with the King of Lineland at first he is simply a disembodied voice coming from nowhere along the line and therefore not perceptible as a figure to his majesty. He therefore lowers himself onto (and through the line) revealing himself as a line as that is all he can be in just one dimension, but a line that can appear and disappear at will. This understanding is vitally important for him to grasp the concept of Spaceland later on in the book when a sphere visits him in his home.

20190416 Flatland 2

As can be seen from the diagram to A. Square the sphere is merely a circle within Flatland and one that can change size and also appear and disappear just as he could in Lineland but even though he had his dream he still struggles to comprehend what it is that he is seeing until the sphere lifts him off the plane of Flatland and shows him his world from above. Suddenly he can see inside his house and not only that but everyone and everything in it simultaneously. He can even see inside his sons, grandsons and servants and also his wife panicking because he has suddenly vanished.

 

20190416 Flatland 4

This is revelatory to him and at this point he grasps a logical progression that had eluded the sphere himself

I. Nay, gracious Teacher, deny me not what I know it is in thy power to perform. Grant me but one glimpse of thine interior, and I am satisfied for ever, remaining henceforth thy docile pupil, thy unemancipable slave, ready to receive all thy teachings and to feed upon the words that fall from thy lips.

Sphere. Well, then, to content and silence you, let me say at once, I would shew you what you wish if I could; but I cannot. Would you have me turn my stomach inside out to oblige you?

I. But my Lord has shewn me the intestines of all my countrymen in the Land of Two Dimensions by taking me with him into the Land of Three. What therefore more easy than now to take his servant on a second journey into the blessed region of the Fourth Dimension, where I shall look down with him once more upon this land of Three Dimensions, and see the inside of every three-dimensioned house, the secrets of the solid earth, the treasures of the mines in Spaceland, and the intestines of every solid living creature, even of the noble and adorable Spheres.

Sphere. But where is this land of Four Dimensions?

I. I know not; but doubtless my Teacher knows.

Sphere. Not I. There is no such land. The very idea of it is utterly inconceivable.

I. Not inconceivable, my Lord, to me, and therefore still less inconceivable to my Master. Nay, I despair not that, even here, in this region of Three Dimensions, your Lordship’s art may make the Fourth Dimension visible to me; just as in the Land of Two Dimensions my Teacher’s skill would fain have opened the eyes of his blind servant to the invisible presence of a Third Dimension, though I saw it not. Let me recall the past. Was I not taught below that when I saw a Line and inferred a Plane, I in reality saw a Third unrecognised Dimension, not the same as brightness, called “height”? And does it not now follow that, in this region, when I see a Plane and infer a Solid, I really see a Fourth unrecognised Dimension, not the same as colour, but existent, though infinitesimal and incapable of measurement? And besides this, there is the Argument from Analogy of Figures.

Sphere. Analogy! Nonsense: what analogy?

I. Your Lordship tempts his servant to see whether he remembers the revelations imparted to him. Trifle not with me, my Lord; I crave, I thirst, for more knowledge. Doubtless we cannot see that other higher Spaceland now, because we have no eye in our stomachs. But, just as there was the realm of Flatland, though the poor puny Lineland Monarch could neither turn to left nor right to discern it, and just as there was close at hand, and touching my frame, the land of Three Dimensions, though I, blind senseless wretch, had no power to touch it, no eye in my interior to discern it, so of a surety there is a Fourth Dimension, which my Lord perceives with the inner eye of thought. And that it must exist my Lord himself has taught me. Or can he have forgotten what he himself imparted to his servant?
In One Dimension, did not a moving Point produce a Line with two terminal points?
In Two Dimensions, did not a moving Line produce a Square with four terminal points?
In Three Dimensions, did not a moving Square produce – did not this eye of mine behold it – that blessed Being, a Cube, with eight terminal points?
And in Four Dimensions shall not a moving Cube – alas, for Analogy, and alas for the Progress of Truth, if it be not so – shall not, I say, the motion of a divine Cube result in a still more divine Organization with sixteen terminal points?
Behold the infallible confirmation of the Series 2, 4, 8, 16; is not this a Geometrical Progression? Is not this – if I might quote my Lord’s own words – “strictly according to Analogy”?
Again, was I not taught by my Lord that as in a Line there are two bounding Points, and in a Square there are four bounding Lines, so in a Cube there must be six bounding Squares? Behold once more the confirming Series, 2, 4, 6; is not this an Arithmetical Progression? And consequently does it not of necessity follow that the more divine offspring of the divine Cube in the Land of Four Dimensions, must
have 8 bounding Cubes; and is not this also, as my Lord has taught me to believe, “strictly according to Analogy”?

Sorry for quoting such a large section but this really is the whole crux of the book as we see that logically there must be a fourth direction that is no more visible to us as up/down was to the square in Flatland and north/south was to the inhabitants of Lineland stuck as they are in their eternal east/west line.

We leave Flatland as we began with A Square in prison for having committed the heresy of declaring of what he calls ‘upward not northward’ and trying to spread these ‘lies’ in Flatland. He is being visited by a priest, as he has been for seven years to try to get him to recant from his madness but instead he determines to write this book.

Flatland has never been out of print since it’s original publication over 130 years ago and it remains one of the great primers in understanding multidimensional geometry so important after the work of Einstein, I heartily recommend it and have thoroughly enjoyed rereading it so thank you Anna.

20190416 Flatland 3