Tom Chi  

What is Trust?

January 7th, 2005 by Tom Chi :: see related comic

There are two types of trust. Intrinsic trust is based on the physical reality of the object. I trust that a rock will fall to the ground when I drop it. This is based on trust in gravity and the property that most rocks are heavier than air. More interestingly to the software world is feedthrough trust which comes from my faith in the designers of a system. This is an amalgam of my faith in their competence, their incentive to treat the consumer well, and their level of accountability for mistakes. I call this feedthrough trust because the trust relationship is truly between established between the system creators and myself, not between the system and myself.

Hardware designers try to build as much intrinisic trust as possible. They want their FPUs to have the property of spitting out proper calculations as surely as a rock has the property of being denser than air. Software is a little hazier. There is certainly infrastructural software which should have this property but day to day we use mostly consumer-facing software. Consumer-facing software does not have this property and in many ways it can’t.

In a software industry where most revenue is built on incremental upgrades, consumer lock-in becomes important. Whether it is Word on Windows or AAC on the iPod, there are a variety of incentives for companies which do not match the needs/desires of the customer. In this way, we as consumers will always feel that software companies have something else up their sleeves which undermines some of our sense of feedthrough trust. Companies can alleviate this somewhat through branding, but it does not change the underlying incentive structure.

Complicating the trust issue are the time dynamics of business. I might give my online information to Plaxo for example and sign and trust their terms of service. If five years from now Plaxo is bought by Company X, my personal information moves along with it, but Company X may or may not respect the agreement I had with Plaxo (it needn’t since Plaxo would no longer be a legal entity).

In reading Microsoft’s Trustworthy Computing whitepaper, Mundie notes that computing systems must reach the level of trust we have in our electrical and financial systems. This is an interesting comparison since electricity is a highly regulated and slow-moving industry. Research in renewable energy sources has been stalled for decades, but the industry is, as noted, stable and trustworthy. This is not where the software industry is right now.

What is possible is for our infrastructural software around privacy and security to become stable and ‘trustworthy’, but this will not make for a scam free world. Taking Mundie’s own example of our financial system — while I have trust that when I transfer money between my bank account to a fund or vice versa it will work properly, the existence of a trustworthy financial infrastructure does not preclude the possibility of people selling me scammy junk bonds or Nigerian millions. Similarly, even when software infrastructure becomes ‘trustworthy’, there will be those who write scammy consumer-facing applications which do bad. This bad will span the gamut from the relatively benign ‘lock-in’ to outright fraud.

The problem is that human intention will always feedthrough in human designed systems, and we have no technology that will ever make all humans trustworthy.

13 Responses to “What is Trust?”
Bob Salmon wrote:

Switching costs: Joel Spolsky has some interesting things to say about this that you might not agree with or have thought of, but are worth thinking about.

Trust: I must confess that where it comes to operating systems and browsers (which it has chosen to make the same thing) I’m afraid that Microsoft still has a mountain to climb as far as I’m concerned. Interesting article on The Register about Microsoft’s purchase of an anti-spyware company, with a nice analogy of toaster makers shipping fire extinguishers.

Bob Salmon wrote:

Sorry if you get some of this twice - my first attempt has disappeared.

There’s an interesting article on Joel Spolsky’s site on switching costs, which is worth a read. (A selling point for your product to new customers is having no switching cost away from it, which takes some guts.)

A friend of a friend had a problem with the infamous dodgy floating point maths in the Pentium. He phoned up Intel to try to get one of the fixed batch, and as they were like gold dust he got the standard “it’s not a problem for most people most of the time so go away”. His reply was “I design oil rigs; do you want to trust your dodgy floating point will be OK?” which was met with a quick “rightio sir, you’ll get a fixed one in the post tomorrow.”

This brings me onto the topic of stakes. Whether something/one is trustworthy or not matters more or less depending on the stakes i.e. the consequences of breaking the trust. The old Spectrum game The Hobbit had a bug in it where if you typed “Open do” at a certain point rather than “Open door” the game locked up. You soon learned not to do this and it was no big deal.

Unfortunately, Microsoft has a mountain to climb for me where it comes to trust, largely due to the stakes involved. The internet is a big bad world (just like meatspace) and all sorts of nasty people will try to defraud me of money, steal my identity, steal control of my machine or just harm its operation in some way (because they’re nice like that). Therefore, it’s really important that the gatekeepers guarding me against all this are trustworthy. Microsoft’s decision to make operating systems and web browsers the same thing, the desire to allow groovy user experiences via email attachments, and the default settings of things go a long way to knock out my trust in them.

It will take a long time to rebuild my trust in Microsoft, so Windows XP2 etc. are great but they need to keep on in this vein for a long time before I don’t smile cynically at the Microsoft adverts in my computer mags that are touting security.

Why will it take so long to trust Microsoft again? I must admit that their business practices sour our relationship from the outset, so that’s a problem peculiar to only some software companies. Apart from that, I think it relates to the stakes again. Because I could lose so much, I need to be very sure that things are OK again. Also, there’s a balance of evidence - the length of their poor track record means they need to show reformed behaviour for an equally long time.

Along these lines, the Register has an interesting article on the purchase by Microsoft of anti-spyware company Giant.

Haddaway wrote:

I think we can learn from a modified version of Haddaway’s wonderful song.

What is Trust?

What is trust
Oh baby don’t hurt me
Don’t hurt me no more
Oh baby don’t hurt me
Don’t hurt me no more

What is trust

Oh I don’t know why you’re not there
I give you my trust, but you don’t care
So what is right and what is wrong
Gimme a sign

As you can see, even Haddaway understood that corporations don’t care much once the trust has been given to them, and that people are very concerned when those they give their trust to come back and hurt them.

TuringTest wrote:

That’s the very reason why Free/Open-Source Software has an economic advantage over proprietary soft in many areas.

You could think that, everything else being equal, software developers would choose to charge some money for their product and thus providing your code for free would be an enterprise suicide.

But open source software has the benefit of trust because you can review it’s inner workings, and you can avoid lock-in so you aren’t tied to the provider. This added value can make an open version of an application widely adopted and therefore a success.

Tom Chi wrote:

There is more than one definition of success though. If a technology is widely adopted by enterprise, but the creators never see a cent, it just doesn’t seem right. The incentive structure is not there for there to be timely releases from the people that know the codebase best.

OSS does make it possible for anyone in the enterprise to make changes to the code and improve it, but branch runs the risk of not being able to take advantage of updates to the core code without significant integration coding/testing. Thus, while in theory organizations should be able to ‘trust’ their ability to audit and add to the code, there *is* sigificant risk in the form of regression and isolation from branching.

In it’s best manifestation, an OSS app is like a peer-reviewed scientific journal. Anything that goes in is carefully read by a team of experienced experts (who are usually paid to do this). This type of system, if carried out for several years, builds a lot of feedthrough trust, and most of the important OSS projects have been some permutation of this system. As a category though, many OSS projects do not work like this. They seem like a pool or random contributors with no one willing to pony up the time to review and build feedthrough trust. Or they sometimes feel like confederations with some sort of loose organization, but also infighting. The problem is the incentive structure.

Without a system to bring monetary incentive to OSS (note you don’t need nearly as much capital to make this run, but you *do* need some), it strikes me that things will continue to somewhat disorganized and slow. In such a system, feedthrough trust is hard to build. Alternatively, you can have a ‘hero’ model, where one or two champions of an OSS app dedicate inordinate time to a project for uncertain gain. There are several projects like this as well, and they produce feedthrough trust (or distrust depending on the competence of the hero). This model is not, however, scalable or repeatable. The type of people with the skills and time to be a competent ‘hero’ are few and far between… and most are already making good money in stable jobs.


Mary Branscombe wrote:

How much trust do we actually have in the financial system? In the UK the clearing banks are now saying that a cheque can fail to clear at any time, so they might take the money back months after you think you have it. ATM security has been poor for years. We trust the financial system because it’s ‘good enough’ and the pain rarely hits the average user. Kind of the same principle as computers?

David Heller wrote:

Such a timely article for me as my company is basically in the trust game. We act as a “firewall extender” for departments that need to give 2nd and 3rd party access during document distribution and exchange processes. This mean trust is key. To me the biggest determinant of trust is professionalism. If someone puts the time, effort, and resources into making something professional looking, well thought after, and well just comfortable, then I will assume that what is behind it is trust worthy. Of course, this is completely perceptual and quite honestly fake.

Let’s assume that everyone has the same level of technology because technology is a commodity. Features of the services are also equal. The only thing that is going to differentiate you is the user experience and how the customer perceives how that ux communicates trust worthiness. Everyplace that the UX fails, it is also failing to communicate that it is trust worthy.

Ethan McKinney wrote:

Given how impossible it is to upgrade our highly customized version of PeopleSoft, I started laughing my butt off when I read this. You’re describing a problem with any customizable software, not just open-source.

OSS does make it possible for anyone in the enterprise to make changes to the code and improve it, but branch runs the risk of not being able to take advantage of updates to the core code without significant integration coding/testing. Thus, while in theory organizations should be able to ‘trust’ their ability to audit and add to the code, there *is* sigificant risk in the form of regression and isolation from branching.

Tom Chi wrote:

Yep. This is true. A lot of enterprise software falls into the customization ‘trap’. Really, the only things that don’t are packaged software and web applications. But I mentioned OSS because the customization trap is pretty much a system-wide characteristic of OSS.

Rob Goris wrote:

How do you know if someone or something is trustworthy? You simply donīt. For most of us trust is based on (1) subjective perception of properties (”the guy telling you he lost his wallet and asks you for a euro for the train looks shabby, I think heīs lying”) and (2) the sanctions you think you can use when the trust is broken (On small islands the crime rates are mostly lower than on the mainland…”if you screw me, Iīll let everybody on the island know!”). Both (1) and (2) are applicable in IT services and software. However I think (2) is getting more and more important, especially on the web. The power of the community has grown dramatically with the web. If somebody feels scammed by a company, heīll let the whole world know. Users know that web con-artists donīt really care about this but companies with a good reputation do. I save money on an internet bank account without any printed evidence or whatsoever and I know the risks but trust them anyway. I remember Nielsen also wrote something on this in an alertbox, canīt find it back though.

Circus Royale wrote:

Trust denote a feeling of certainty that a person or thing will not fail. Trust implies depth and assurance of feeling that is often based on inconclusive evidence.
Faith connotes unquestioning, often emotionally charged belief.
Confidence, frequently implies stronger grounds for assurance.
Reliance connotes a confident and trustful commitment to another.
Dependence suggests reliance on another to whom one is often subordinate.

Bob Salmon wrote:

Can trust (or trustworthiness) by measured? For instance, the standard measurement of beauty is the milli-Helen (the amount of beauty required to launch one ship).

What would an appropriate unit of trust be?

  • Distance? (”I wouldn’t trust him as far as I could throw him”, which would suggest multiples of this, although it’s relative to the speaker.)
  • Value of a possession with which you could trust the person/thing? (This raises another question: how do you define the value of something like your son?)

Maybe this isn’t as simple as that. You could trust someone to make a mistake (and if you’re a devious soul use this to your advantage) but not trust them to tell the truth. Is trust predicated on the person/thing and the activity/scenario, rather than just the person/thing?

For instance, I’d trust Microsoft to come up with a decent user interface to their products (at least relative to their peers). I wouldn’t trust them to conduct their business with ethics. I therefore cannot say a simple “I trust Microsoft” or “I don’t trust Microsoft”.

"kemi" wrote:

“while I have trust that when I transfer money between my bank account to a fund or vice versa it will work properly, the existence of a trustworthy financial infrastructure does not preclude the possibility of people selling me scammy junk bonds or Nigerian millions.”

Dear writer,

I was just reading your report and came acros the above sentence. I am Nigerian and I find your singling out Nigeria as a culprit of this offence is totally unfair.

While I recognise, and there is evidence to prove, that there are a lot of “scammy junk bonds” which claim to originate from Nigeria, there are other African (West and East) countries who I and other people I know receive these from. There are documentaries which highlight these other countries.

I also receive a lot of fraudulent spam from countries like Australia, Spain, India and recently from people who claim they are legitimate representatives of a certain UK bank from Hong Kong and from Ebay. I have also been defrauded by a seller from an asian country on Ebay, thankfully not a lot of money which was refunded back to me.
So to single out a country or not to mention other countries that are also guilty of this in your report is in my opinion not right. I hope you can see sense in this and put straight your report.

Kind regards,

Leave a Reply

OK/Cancel is a comic strip collaboration co-written and co-illustrated by Kevin Cheng and Tom Chi. Our subject matter focuses on interfaces, good and bad and the people behind the industry of building interfaces - usability specialists, interaction designers, human-computer interaction (HCI) experts, industrial designers, etc. (Who Links Here) ?