The EMV standard has long supported an offline transaction flow. AFAIK it was the default almost everywhere in Finland circa 2011, contactless there was almost always instantaneous. Digging into why that was compared to the invariable wait when using contactless in the UK revealed this flow.
The card has a variety of risk counters on it that allow it to securely decide whether an offline transaction can proceed, at least some of which are also exposed to the terminal which can have its own separate policy. I imagine internally the banks and payment gateways have a huge variety of internal related tuneables.
Yes the EMV standard has offline card payments baked into it, and most banks in the EU and UK used to use offline transactions for contactless.
Your experiences in the UK are almost certainly linked to the card issuer you were using (was it a Monzo card by any chance), and nothing to do with it being the UK. The vast majority of the legacy banks have always used offline transactions for contactless.
However there has been a bit of shift towards online transactions, driven by EU rules likes strong customer authentication, which requires regular pin entries determined by cumulative spending and duration limits (which ever is hit first). It’s a lot easier to reliably meet the requirements of SCA using online transactions.
As for how offline transactions work. It’s reasonable simply. The terminal asks the card to sign the transaction using the cards private key. Now there is an extremely complicated set of rules around how liability shifts in the event of a fraud claim, depending on many factors like the type of transaction, if a pin was entered and validated by the card, if the card ask to go online and the terminal ignored the request, they type of merchant, the exact region your in etc etc.
But regardless of all that nonsense. The technical process is very simple. The terminal has the transaction cryptographically signed using symmetric encryption with a private key that is only known to the card and the issuer of the card. That signed transaction can later be presented to the issuer so the merchant gets paid.
Given it’s a symmetric key, you may wonder what happens in the event of a dispute between the issuer and the merchant, where the issuer claims they received a forged transaction. To which the answer is, the issuer sends a signed and sealed letter to card network operator saying they have double checked the transaction signature, and believe it to be forged. And if anyone doesn’t believe them, they can sue em (this is not at all a joke, it’s literally the documented and contractual process used by the major banks and card networks).
> However there has been a bit of shift towards online transactions
Not just a bit of a shift: Offline-preferred transactions are basically a thing of the past. EU rules have nothing to do with it (offline EMV is fully SCA compliant as well, as the chip can keep count, although it's a bit annoying to keep counters in sync if there's more than one); it's mostly for risk management and simplification reasons, I believe.
> The terminal has the transaction cryptographically signed using symmetric encryption with a private key
Offline transactions essentially always require card authentication as well, which requires asymmetric cryptography. (Otherwise, you could trivially forge valid-looking cards and offline terminals would be none the wise since they obviously don't hold the symmetric keys for every card issued; that would be way too risky).
> EU rules have nothing to do with it (offline EMV is fully SCA compliant as well, as the chip can keep count, although it's a bit annoying to keep counters in sync if there's more than one);
EU rules have everything to do with it. Meeting EMV with offline counters is tricky to get right. The reason I can confidently claim the SCA drove the migration of online transactions is because I was responsible for the technical implementation of SCA for a bank, and was part of the industry wide conversations that happened as banks figured out who they would comply with the rules, and tried to figure out what wiggle room existed. So I know SCA was a driver, because I talked to the people making this decisions.
> which requires asymmetric cryptography. (Otherwise, you could trivially forge valid-looking cards and offline terminals would be none the wise since they obviously don't hold the symmetric keys for every card issued; that would be way too risky).
This is where you are simply wrong. Again speaking as someone with actual experience working on this stuff, and having dealt with actual forged offline transactions as well. I can tell you the card uses symmetric encryption, the terminals themselves have zero ability to validate the signature a card produces. The terminals include some data that allows them to validate if a card number is routable, and also a hot list of card numbers to always reject, and thus reject transactions that can never be fulfilled, but they have no way of validating that a specific card or transaction hasn’t been forged.
The entire system depends on the physical security of the chips in cards to make key extraction extremely hard, and to also make the correct mimicking of cards extremely hard. But it’s hardly impossible, and there’s plenty of demonstrated EMV attacks out there in the wild. It’s just that they all hard to replicate, require special equipment, and generally make you look very dodge in the moment because you have to have a weird card with wires running your shelves so you can intercept the card comms. The manufacturer and distribution of the chips in cards is extremely tightly controlled, so you can’t just buy them. In theory anyone with a million dollars and ASIC contract could manufacture their own ICs, but there simply easier ways to steal money.
Ultimately EMV only needs provide enough security to make other methods of stealing more palatable. Cards are used for pretty low value transactions, so breaking EMV isn’t a very scalable way to steal money. You would be better off stealing products from merchants using decent pair of running shoes, rather than creating crazy ways to trick their payment terminals. Running shoes are cheaper and much more accessible than ASIC production.
> The reason I can confidently claim the SCA drove the migration of online transactions is because I was responsible for the technical implementation of SCA for a bank
Ok, "nothing to do with it" was too strong: I don't doubt that SCA was the death knell for many offline implementations. However, I've seen online-only cards in the field by many banks well before SCA became effective.
And on the other hand, I also know SCA-compliant EMV implementations that do still support offline transactions. As you say it's tricky, but it's possible. Europe is large, and you probably haven't talked to every single bank/processor :)
> This is where you are simply wrong. Again speaking as someone with actual experience working on this stuff, [...] they have no way of validating that a specific card or transaction hasn’t been forged.
Huh? If you have worked on this stuff, surely DDA and CDA ring a bell? They're both based on asymmetric cryptography, and they absolutely allow the terminal to dynamically verify whether a given card is authentic or cloned, without having to go online.
Without that, you could indeed copy the static signature data from any EMV card and replay it to an offline terminal and get away with it. That's why SDA has long been deprecated for offline transactions.
> The manufacturer and distribution of the chips in cards is extremely tightly controlled, so you can’t just buy them. [...] In theory anyone with a million dollars and ASIC contract could manufacture their own ICs
You can absolutely get them on Aliexpress straight from the manufacturer for a few bucks each, cheaper in bulk. They run Java, so you can just write your own software – the EMV specs are public!
What you can't get there is the cryptographic private key specific to the card number that the issuing bank embeds into it at personalization time.
EMV is a cryptographically sound (if dated and very complex) scheme, and secrecy of implementation is actually much less of a security factor than you seem to claim, despite the industry's (largely historical) obsession with secrecy.
> Huh? If you have worked on this stuff, surely DDA and CDA ring a bell? They're both based on asymmetric cryptography, and they absolutely allow the terminal to dynamically verify whether a given card is authentic or cloned, without having to go online.
Yes I was a little wrong here. My most recent experience in this area is dealing with messages on the issuer side. It’s been a while since I’ve done anything serious at the card config level, and don’t have a huge amount of experience with the handshakes that occur between card and terminal.
The missing nuance is that the transaction cryptogram is symmetrical encrypted, and that’s the only cryptographic blob sent over the card network to the issuer. As the asymmetric stuff only happens locally between card and terminal, and isn’t included in any data transmitted from terminal to issuer.
So the terminal can check that the card is a real card using asymmetric encryption, and that the produced transaction cryptogram was produced by that card. But none of that evidence is passed on to the issuer, only the symmetric cryptogram, which can’t be used for non-repudiation is sent to the issuer later.
Hence the experience dealing with forged transaction cryptograms, which were forged by an acquirer in a systematic manner to try and gain stronger chargeback protection because their sloppy transaction processing resulted in them processing a lot of fraudulent transactions, and then being hit with a lot of chargebacks. The process of “proving” to the network that this acquirer was forging cryptograms was sending a letter signed by legal team attesting to fact we had evidence of forgery, plus a much nastier letter sent to the acquirer in question to knock it off, and stop doing obviously illegal things. Ultimately it’s the acquirers legal counsel that acts as the enforcement mechanism here, and the obvious threat of lawsuit they’re clearly can’t win tends to be a very strong motivator for companies to tidy up their act.
> EMV is a cryptographically sound (if dated and very complex) scheme, and secrecy of implementation is actually much less of a security factor than you seem to claim, despite the industry's (largely historical) obsession with secrecy.
I may have overstated it a little. Bad habits from spending too much time dealing with PCI rubbish. The difficult thing to get across in these threads about payment networks is how much of the systems “security” really comes from clever legal contracts, and smart distribution of liability and risk. Effectively making in everyone’s interests to not do anything really silly, but the actual technical security is almost secondary to legal mechanisms that exist to ensure that participants are highly motivated to make sure that fraudulent transactions are kept to a minimum.
Ah, so you're talking about the edge case where the terminal claims to have, but did not actually, perform ODA? Yes, that's somewhat of a gap in the EMV protocol. As I've mentioned in my other comment, I could see CDA eventually becoming mandatory, as well as keeping the entire CDA output terminal-side. That trace does provide non-repudiation.
There are other ways too to stop "sloppy terminal processing", but as far as I understand they're not cryptographically secure in a way that would provide an unambiguous and third-party verifiable protocol trace.
I suspect that all of that is a big reason why the networks don't love offline processing if it can be avoided.
And I couldn't agree more to your last paragraph – the industry does have an unfortunate history of propping up questionable security engineering with legal threats. But I'm slightly more optimistic on EMV, at least some implementations: Decades later, we can actually have some nice things :)
Also, thanks for the anecdote! Helped me confirm a theory I had on the motivation for a particular obscure protocol feature that I have so far not found solidly explained anywhere in the literature.
I used to have an online maestro card (was solo and now known as debit mastercard) and an offline card (was switch, now also known as debit mastercard) from a UK bank, due to having two current accounts there.
The offline card was from a current account with an overdraft and also worked as a cheque guarantee card, for cheques up to £250 under the (discontinued ~2011) cheque guarantee scheme[0] and had a special hologram on the back. The retailer would watch you sign the cheque and write details about you, the card and any CCTV etc. on the back of the cheque. I imagine the offline behavior of the card was similar, and was a carry over from that.
The online card was from a basic account with no overdraft facility and acted a bit like a prepaid debit card.
The tech is fundamentally fine - where the problem arises is human behaviour.
When offline contactless payments first emerged in the U.K., there was a significant spike in unplanned overdraft usage and disputed transactions, as people rapidly realised it essentially worked as a free line of credit, up to £100 or £250 depending on your issuer.
This quite quickly caused issuers to push up prices on offline transactions, which made them less appealing to merchants - add to that that people were talking about them being a hotbed of fraud, and merchants err away from offering them, PCNs start dropping the capability due to lack of demand, and here we are today.
I think metro trains in countries like The Netherlands and Singapore use this approach where in you tap your cards and entry and exit and you are billed usually somewhere at the end of the day.
Yes, stored value cards are a very mature technology.
For various reasons, they're mostly used in closed-loop systems these days (think laundromats, transit systems etc.), but historically there were open-loop deployments in many European countries, and in some countries, stored-value POS payments are still very popular, e.g. in Japan.
It's a real shame that the entire world moved to online-only. Sure, it's much easier and there's less opportunity for various kinds of fraud as a result, but in terms of availability during outages or cyber attacks, it was a big unforced step backwards.
> historically there were open-loop deployments in many European countries
Indeed, there used to be things like "Moneo". The problem is that banks never trusted really these systems, so you were limited to, say, 50E of stored value. Also for some reason in Europe the readers of such cards have never been great, I guess because most devices were built on the cheap, so even if the transaction is offline and supposedly fast, you would have to wiggle your cards all around most readers for 2s until it's picked up.
In Hong Kong there is the Octopus card, which started as a closed loop subway card, but ended up being so loved that now you can pay litterally anything with it. It can store up to $500, and you can set it up to automatically top-up to $500 more per day linked to your bank account. Also accepting payments from octopus cards is very easy, you don't need a physical device and small businesses can just have the customer card tapped on their phone with a merchant app.
These were storage only though, right? Such systems are trivial to compromise.
Stored-value payment cards usually contain at least a secret key and some logic that allows them to establish a secure channel to another trusted entity, such as a merchant smartcard (which can be embedded in a terminal) or a backend server (and a corresponding HSM).
Not in the Netherlands, no. The card can be a bank card, and you can be billed at the end of the month automatically through direct debit.
It also wouldn't work as you describe, as the terminal at the point of entry doesn't know how much to charge you since it doesn't know where your journey ends.
Some transit systems work by putting a hold on your card for a nominal amount. When you finish your journey it then only claims the cost of your journey
Holds don't really show up in the monthly statement. At least not in the cards I've had. It's a functionality for merchants to say "I'd like to charge this customer up to $500, would she be good for it?". If the CC company says yes, then the merchant knows they can do so. E.g. car rental companies do this for potential damages. Up to a week the merchant can charge the actual amount (usually less) or just release the hold.
Holds are a credit card feature, GVB is a Dutch transit authority, so they're more likely to be talking about bank cards, ie. debit cards, which I don't think support holds in that same sense.
It will bill your 4 EUR (on a tram/bus) or the whole 20 (or something on a train) instead of the actual journey price if you forget to checkout. Pretty sure it can decline cards for insufficient balance too. Not sure the entry gate blocks the amount.
Actual chipcards don't bill you at the end of the month either -- they reload a fixed amount through direct debit (which takes a few days) the moment your balance crosses zero. If the direct debit isn't setup for a card (because it's not a personalized card) or the debit was rejected, the card is blocked.
For business chipcards cards it works somewhat the way you described.
It definitely exists in Amsterdam, no? When I visit I just tap my card/phone on NFC enabled card checking machines (which exist in and out of the stations)
A lot of internet payments work this way already anyways, not many gateways require auth before capture, processors/payfacs just do it because it gives lower interchange and reduces risk.
That depends on where you are. But all the major networks expect auths before capture, even if it isn’t technically enforced, and will punish gateways and merchants who have low auth rates.
Auth before capture doesn’t generally reduce interchange. What it primarily does is shift liability in the event of a dispute. If a chargeback is raised, and no auth happened, then the merchant simply looses immediately. They have no mechanism for fighting the chargeback. If they auth first, and got an approval, then it’s the banks problem in the event of a customer dispute. The merchant can reply to the chargeback by pointing out the valid auth they received from the bank, and the bank has to go pound sand.
You may see this as different “interchange” rates from a specific gateway. But that’s simply not true at the network level. The difference in pricing just exists so the gateways themselves can price in the additional risk associated with auth less captures, given the gateways are always on the hook, even if a merchant goes bust. The major networks force gateways to have funds kept in escrow that are guaranteed to cover any shortfall that might occur due to individual merchant failure, or failure of the gateway itself. That how networks make sure that zero real risk every accrues with them, they make everyone else put up huge stacks of cash to ensure that every virtual cent that’s in flight at any moment, is backed by a real cent in escrow somewhere.
I always wondered how come that some online merchants where I record the card do not ask for auth or 2 factor and some other does. For example, I have recorded my card info in a webapp that I use to pay the bills, and it never asks for anything, which is good, as I have many bills to pay, and I wouldn't want spam of 2FAs. Can you provide some technical terminology on this behavior or links, since you seem to be in the knows. Thanks.
Amazon as well seems to never do proper auth (at least here in the UK). When using an existing card where the delivery address matches the billing address, card transactions go through instantly
There is no rule that there has to be a single payment per authorization, if you read for example mastercard API https://in.gateway.mastercard.com/api/documentation/integrat... it says you can partially capture, you can extend authorization. So for all you know when you buy at amazon it just updates the authorization and charges you later but keeps authorization going and amazon might just check if they have auth token for your card and if they do system allows you to continue and to you it seems instant.
It’s a lot more complicated and nuanced than that. The API docs you link too are for an API that covers a tiny fraction of what can be expressed in the actual ISO 8583 messages which are the real “API” of the card networks. The docs for those are hundreds of pages long.
Plus you need to analyse how the different messages types and sequence of messages interact with the transaction processing rules, which are also hundreds of pages long.
Suffice to say, the entire system is insanely complicated, and just about everyone out there implements it all incorrectly, with the whole system on working because partners are only allowed to complain about the insanity if they actually loose money. Until that point they’re expected to just handle everything as best they can.
Ultimately all this stuff comes down to risk and risk management. The card networks themselves don’t provide any technical enforcement of most of their rules, beyond low level technical rules around message format and structure (and even that enforcement is pretty small).
Instead they effectively make all the parties that connect to the network responsible for rule enforcement. If a merchant follows all the rules correctly then they receive extremely strong chargeback protection, I.e. if an issuer sends a chargeback, and the merchant has plenty of grounds to dispute the chargeback and win.
If merchants don’t follow all the rules, then issuers can send chargebacks, and it’s much harder for the merchants to defend themselves.
In all scenarios it up to the issuers and merchants to explain in the chargeback what rules have been broken, by which party, and thus who should win based on network transaction rules. The networks themselves don’t even make a ruling directly, instead the issuer and merchant decide who wins via a back and forth process that includes escalating fees paid to each other, until one side gives up. The networks only get involved the two sides can’t resolve the issue themselves, and will charge the looser a significant fee for the privilege, so there’s a strong incentive for the parties to resolve the issue themselves.
How does of this interact with 2FA, auths etc etc. Basically 2FA, and ordinary auths are all just things a merchant can do, or trigger, to reduce their liability and get better chargeback protection. If the merchant performed a full 3DS auth, where the issuer is asked to perform 2FA, then they have pretty complete chargeback protection in the event of fraud, because they’ve basically asked issuer to make absolutely 100% that this transaction has been approved by the issuer’s customer, so there’s zero grounds later to claim that a stolen card was used or something similar. If the issuer’s customer wants to dispute the transaction, that’s the issuers problem.
But all of these mechanisms reduce checkout rates, and thus merchant revenue. As a result some extremely large merchants make a trade off of basically accepting all the risk of fraudulent transactions, and give up chargeback protection, but not following all the rules. The merchant does this because they’ve basically asked believe they have enough data to prevent fraudulent transactions, without using any of the tools the card networks provide.
For merchants that can do this (like Amazon), they build in-house fraud detection systems, and payment systems that evaluate the risk of each transaction, then change the exact way they perform the transaction to either reduce friction (because the transaction is very low risk) or increase friction (because the transaction is higher risk), thus allowing them to capture more revenue, without taking on more risk (because they have confidence their ability to detect fraud, and thus don’t need help from the issuers).
But there are very few merchants that can even do this, as it generally requires either a very collaborative payment gateway (who are ultimately on the hook for merchant misbehaviour), or a direct connection to the card networks (who aren’t interested in talking to people not moving millions of dollars every day). Which is why it tends to pretty rare.
The way gateways and networks handle auth are not the same thing and it gets really muddy and confusing, honestly.
If you’ve already tokenized the card on a gateway for a particular merchant, for example, they may allow you to keep pushing multiple charges while on their end still using the original network auth from the first tokenization - which ends up being entirely opaque to you, the client of the gateway.
Essentially you don’t have to care what the card network rules are, just how your gateway presents functionality to you.
The card has a variety of risk counters on it that allow it to securely decide whether an offline transaction can proceed, at least some of which are also exposed to the terminal which can have its own separate policy. I imagine internally the banks and payment gateways have a huge variety of internal related tuneables.