A few days ago I wrote about using PushKit as an alternative to real-time connections to save battery power.
I decided to do a small test over time to see what the delivery times for a PushKit were. The results where both impressive and strange.
Here is the set up. Every 30 minutes I will send 32 push messages to two handsets. The first two messages are warm-up messages to get the connection to Apple servers up an running. I am just too cheap to keep my VM up and running at all times 😉
Every push message included a timestamp when it was sent from the server. Since PushKit allows you to run code in background (thank you Apple), the app pushed that back to the web service, so in effect I got a total round-trip time for each push.
Since I didn’t measure only Apple push kit performance, due to the fact that I am doing round trip measurement, I want to disclose my setup.
I am hosting a web job on Azure and calling a web service to send pushes, and receive reports back from the handsets.
It’s a basic instance not always running in West of the USA. Regular network latency to this service for the phones is 150ms from the home wlan.
So when you look at the numbers below, 150ms seconds is for reporting back the numbers.
After running it for a day and 2370 push messages, here are the results
|Average (ms)||Max time (ms)||Min Time (ms)|
Nothing catastrophic: It’s not like a live open socket your iOS and the numbers seemed to be very scattered so I made a breakdown of average time of day. The most horrifying number is a max time of 61 seconds to deliver.
Now apart from a couple of spikes during the day, we seem to have a response time of less than a second except during the night. I suspect that my 2am–8am response times is Apple putting my phone on low power consumption because it learnt that I don’t use the phone that much during those hours. But it is of course Apple, so you just don’t know.
Let’s have a look at the two spikes at 11am and 8pm. Looking at the spike for 8PM I found the max value, 61150 occurred 20:51:36, and the message was to my phone. I live in an apartment building where the data connection is really poor and as it happens, I know I went out to meet Rinse for a pickup of laundry. Taking a closer look at all the pushes around that time, there are also 12 missing messages. So for that particular scenario I don’t think I can blame Apple, it is just my data connection.
Let’s take a look at 11AM. Same thing here, I was going out to the store and ended up with a bunch of 20+ seconds push messages as I was walking down the hallway.
Now I have a choice of simply removing the extremes, or maybe just look at my test phone. I want to test live usage so let’s keep them. But out of interest removing all over 20 seconds will give you an average turnaround time of 1138.
Let’s have a look at the distribution of response times. Now this looks more encouraging, almost 75% under 700ms and ~90% are under 3 seconds.
For you that don’t want the numbers skewed by real life usage of a handset (such as not being online etc.), I took a look at 121 messages over 7 seconds, they are all lumped together and correspond to times I knew I was not in coverage.
|Average||Max Time||Min Time|
As you can see, that improves the average a whole lot. But remember that this is not from a phone in someone’s pocket.
So far it has only lost 40 messages during the 24-hour test, and they seem to lump together while I was walking out of the building. Overall I think that you can trust that messages will be delivered. One worry is of course the timing and queue time Apple has. It seems to be quite a short time queued before it gets thrown away.
Pretty damn impressive in my opinion. This is only on their sandbox, and my backend is running on a basic Azure backend that is not always running. But for VoIP calling, getting the signal under 1 second 90% of time is pretty solid. A signalling time under 3 seconds for setting up a call is in my view acceptable. It’s just like the regular phone network.
I am going to let this test run for a couple of more days to see what it does to the numbers. It will be particularly interesting to see if I get more lost messages.
We’ve got a new tutorial for you! Our resident petrol head Christian Jensen has been on a steep learning curve, hanging out at racetracks and coming up with a really cool way to use SMS to manage signups and… read more
Since 1957, when a five-year-old boy with perfect pitch first phreaked AT&T switches and invented phreaking, phones have been a target for different types of fraud that costs customers and phone companies billions of dollars. However, if you’re using the… read more