Internet Web Help

Good article in The Economist that looks at the wide range of datamining activity on Social Nets – firstly, it breaks marvellously benign new ground:

..broadening data mining to include analysis of social networks makes new things possible. Modelling social relationships is akin to creating an “index of power”, says Stephen Borgatti, a network-analysis expert at the University of Kentucky in Lexington. In some companies, e-mails are analysed automatically to help bosses manage their workers. Employees who are often asked for advice may be good candidates for promotion, for example.

Crime can be reduced….

Ellen Joyner of SAS, an analytics firm based in Cary, North Carolina, notes that more and more financial firms are using the software to uncover fraud. The latest version of SAS’s software identifies risky borrowers by examining their social networks and Internal Revenue Service records, she says. For example, an applicant may be a bad risk, or even a fraudster, if he plans to launch a type of business which has no links to his social network, education, previous business dealings or travel history, which can be pieced together with credit-card records. Ms Joyner says the software can also determine if an applicant has associated with known criminals—perhaps his fiancée has shared an address with a parolee. Some insurers reduce premiums for banks that protect themselves with such software.

The police department of Richmond, Virginia, has pioneered the use of network-analysis software to predict crimes. Police officers know that crime increases at certain times, such as on paydays and when there is a full moon. But the software lets them analyse the social networks around suspects, such as dealings with employers, collection agencies and the Department of Motor Vehicles. The goal, according to Stephen Hollifield, the department’s technology chief, is to “pull together a complete picture” of suspects and their social circle.

Party plans turn out to be a particularly useful part of this picture. Richmond’s police have started monitoring Facebook, MySpace and Twitter messages to determine where the rowdiest festivities will be. On big party nights, the department now saves about $15,000 on overtime pay, because officers are deployed to areas that the software deems ripe for criminal activity. Crime has “dramatically” declined as a result, says Mr Hollifield. Colin Shearer, vice-president of predictive analytics at SPSS, a division of IBM that makes the software in question, says it can largely replace police officers’ reliance on “gut feel”.

Secondly, it finds the real influencers, good and bad:

TELECOMS operators naturally prize mobile-phone subscribers who spend a lot, but some thriftier customers, it turns out, are actually more valuable. Known as “influencers”, these subscribers frequently persuade their friends, family and colleagues to follow them when they switch to a rival operator. The trick, then, is to identify such trendsetting subscribers and keep them on board with special discounts and promotions. People at the top of the office or social pecking order often receive quick callbacks, do not worry about calling other people late at night and tend to get more calls at times when social events are most often organised, such as Friday afternoons. Influential customers also reveal their clout by making long calls, while the calls they receive are generally short.

…………….

Network analysis also has a useful role to play in counterterrorism. Terror groups are often decentralised, so mapping their social networks is akin to deciphering “a big spaghetti picture”, says Roy Lindelauf of the Royal Dutch Defence Academy, who develops software for intelligence agencies in the Netherlands. It turns out that the key terrorists in a group are often not the leaders, but rather seemingly low-level people, such as drivers and guides, who keep addresses and phone numbers memorised. Such people tend to stand out in network models because of their high level of connectedness. To find them, analysts map “structural signatures” such as short phone calls placed to the same number just before and after an attack, which may indicate that the beginning and end of an operation has been reported.

Marvellous, I hear you say – what can go wrong? Well, nothing except the amount of data about you that they want, and all the other things they can predict with it – like your infidelities for example (believe me, you can…). But it is not going to go away:

The market for such software is booming. By one estimate there are more than 100 programs for network analysis, also known as link analysis or predictive analysis. The raw data used may extend far beyond phone records to encompass information available from private and governmental entities, and internet sources such as Facebook. IBM, the supplier of the system used by Bharti Airtel, says its annual sales of such software, now growing at double-digit rates, will exceed $15 billion by 2015. In the past five years IBM has spent more than $11 billion buying makers of network-analysis software. Gartner, a market-research firm, ranks the technology at number two in its list of strategic business operations meriting significant investment this year.

And its getting easier – 5 years ago I needed all I’d learned in an MSc in Engineering doing what what was effectively Stats and Operations Research, but now:

A decade ago IBM employed experts with PhDs in mathematics to study social networks, according to Mark Ramsey, the firm’s head of business analytics for eastern Europe, the Middle East and Africa. Today, college graduates can operate analysis software handling enormous quantities of data. Bharti Airtel employs only about 100 analysts to keep tabs on its 135m subscribers.

I was at an early futurology session on this about 10 years ago, the endgame was succinctly described as being able to predict the “Net Present Value of your Future Spend”.

You have been warned……..

Worrying news from Ars Tech re Journal of Consumer Research* paper on making people hand over privacy data – we are not ratinal and and over more intimate details to (probably) riskier sites:

The researchers set up two survey web pages, one of which looked very official: it had the Carnegie Mellon University seal, and referred to a “Carnegie Mellon University Executive Council Survey on Ethical Behaviors.” The other, well… Comic Sans featured heavily in the site design, and the survey page was entitled “How BAD Are U???” In a pre-test, far more people rated the official-looking page as a safer option for transmitting personal information.

When put to the test, however, the exact opposite occurred. Depending on the question, participants who used the How Bad ARE U version admitted to unethical or embarrassing activities at a rate of 1.74 to 1.98 times that of those who were given the professional version. In a separate survey, participants rated the same questions as less intrusive if they were presented in Comic Sans—even though there was no difference in the ratings of the activity’s social desirability between the two survey populations. In short, an unprofessional-looking interface seemed to loosen participants up in the same manner that approaching a question indirectly did.

Also…..

[The researchers] collaborated with The New York Times to create a web survey entitled “Test your ethics,” which asked participants to rate the ethicality of a set of actions. But, in the process, users were asked to indicate whether they had ever engaged in those activities, under the pretense that it might color their ratings.

Answers varied a great deal based on the perceived intrusiveness of the question, but one pattern became clear: it was possible to get more people to answer that they had engaged in a given behavior if their own behavior was approached indirectly. If participants were asked about their participation as part of the rating process, they were about 1.5 times more likely to admit an ethical misstep than if they were simply asked point blank as a separate question. This suggested that a casual approach, which puts a participant at ease, is more likely to get them to cough up personal details.

Add to that the way gaming reward functionality is increasingly used (Gaming rewarsd have been shown to be effective at getting people to divulge stuff as they rewad them for it) and you have a perfect culture for privacy raiding.

*There is no link to the paper unfortunately, so I haven’t actually read it. Ars Technica is one of he more sanguine blogs however.

Both Forrester and Ars Tech try and explain why the Intel/McAfee deal is being done:

Forrester:

1. This is not just about “antimalware-on-a-chip-for-smartphones”. Another side-effect of people not understanding the deal is that they oversimplify it by reducing it to this one aspect…… this is about a wide range of embedded security features (not just AV, but data security and system integrity) on a wide range of devices.

2. Intel needs McAfee to thrive as a software business in its own right. In some areas, embedded security will be fully contained In the chip/system: such as Intel’s XD bit technology.

3. We need to look beyond Wind River as a model for how McAfee/Intel should evolve….. The wisdom of the Intel-McAfee deal and long-term success of McAfee will hinge both on future acquisitions and support of its growth as a stand-alone security business and also on Intel’s ability to combine McAfee technologies and its own in new ways.

4. Anti-trust concerns are avoidable or surmountable.

5. The price is quite reasonable. McAfee….. is at a 3.4x multiple. Compare that to other security acquisitions …. such as Websense’s 2007 acquisition of Surf Control (3.8x), Check Point’s 2007 acquisition of Pointsec (8.8x), or IBM’s 2006 acquisition of ISS (3.2x).

Ars Tech:

Security is Job One

At the most recent Intel R&D day, Intel CTO Justin Rattner did a Q&A session with the press in which he was asked something to the effect of, “What do you spend most of your time working on these days?” Rattner didn’t hesitate in answering “security.”

Moving up the stack, and then off the stack

Intel’s years of experience with vPro and its predecessors have no doubt confirmed to the company that providing silicon-level support for advanced security and remote management technologies is a waste of time if no systems integrator or popular software vendor implements them in some kind of consumer- or business-facing product or service.

Why they did it

In explaining its purchase of McAfee, Intel has clearly indicated that the real impact of the purchase won’t really be felt in the computer market until later in the coming decade—this is a long-term, strategic buy. This statement fits with the idea that acquiring McAfee is Intel’s way of bringing vPro and subsequent security efforts directly to businesses and consumers by just buying out the middle-man. The McAfee purchase gives Intel an instant foothold on countless PCs, a foothold that Intel itself would have to spend years building (if it were even possible).

I’d be happier if (i) they agreed with each other and (ii) that word “strategic” didn’t keep popping up (I ciulled a lot of the text in both articles, so you can’t see it but you can get the sense even from what I pasted up).

In other words I still don’t think anyone really knows what is going on.

Intel bought McAfee fo 60% over share price today – GigaOm:

Intel CEO Paul Otellini said in a statement: “In the past, energy-efficient performance and connectivity have defined computing requirements. Looking forward, security will join those as a third pillar of what people demand from all computing experiences.” The price Intel agreed to pay for McAfee — which had revenue of $2 billion in 2009 and has gross profit margins in the 80-percent range — is a 60-percent premium to its trading price prior to the announcement.

Quite – its all about The Cloud – but McAfee? And a 60% premium – $7.7bn for the thing, in cash – at exactly the time when many newer, cheaper (and dare I say better) security software companies are popping out the woodwork? Elsewhere its been justified as a deal “about mobile” but I don’t get that either for the same reason as above.

I rather liked a comment on TechCrunch:

It’s simple really. McAfee’s software slows down your computer enough that you need a faster Intel CPU. Intel now has direct influence over the main driving factor behind people purchasing new PCs…

That is about the only way this makes any sort of sense to me at the moment. To be watched…..

Rather interesting observation yesterday – I was busy with client work so couldn’t attend the Grauniad Activate 2010 conference, so I dipped in (as is my wont) to the User Generated Twitterstream (#activate2010). At the same time there was a Guardian liveblog and a moderated Twitterstream from its Journalists was going on.

What was interesting to me was the massive degradation in the User Generated Twitterstream. Last year, and early this year, you could tune in to such Twitterstreams and get a fairly decent “user generated media” view of what was going on. The “User Generated” Activate Twitterstream yesterday was….well, “unhelpful” would put it mildly.

Key issues I spotted were that retweeting of content was far more focussed than before, but not in a good way – the main focus was on:

(i) Uncritical mass retweeting of “soundbite sayings” – “Kool aid” homilies etc

(ii) Similar mass retweeting of very dubious statistics, again totally uncritically.

(iii) In fact my impression overall is that the Twitterstream was becoming an “empty vessels making the most noise” mode of communication. The Retweeting seemed more about marking cyberterritory by pissing on the digital lamp-posts than actually communicating anything.

The disparity between the Twitterstream and the Grauniad “Journalist Generated” Liveblog and Twitterstream became more and more marked as the day wore on, leading me to ormulate two hypotheses for “User Generated Media” going forward:

(i) The “Citizen Reporter” on the twitterstream, who was a pretty reliable eyewitness 2007-9. is increasingly being drowned out by flacks, fanbois and noisy numpties. “Proper” mainstream media journalism again is becoming a far more trusted source. The age of “You Media” is over.

(ii) I eventually stopped watching the Twitterstream, setting up a temporary stream from a group of people whose views I trust that were there. In other words, the Twitter Signal to Noise ratio is gettng to the point where filtering the folksonomy by trusted source will be essential.

The obvious lesson for presenters from yesterday is that using mental bubblegum soundbites that send the twitternumpties into RT frenzy is a far more effective way of getting your message across than any reasoned argument. That was ever thus, true – but the only danger with this, I would argue from my reasoning above, is that over time it degrades to lowest common denominators sans moderation – a tragedy of the commoners, as it were. It will work like online advertising – you do it to get to the most influential people, but they are the first to avoid it – so you wind up banging your gong to the digital peasantry.

And on the way, the countless retweeting empty vessels toll the death bell for Citizen Journalism on Social Media platforms.

No what surprises me a lot about this post is that I wrote it! I’m no friend of the MSM, and a fan of the whole “2.0″ thingy in general, but what this showed me is that, left to its own unfiltered devices, “User Generated” content is a lot like a the h3nry Sewer of Life – what you get out of it depends on what you put into it. And like that sewer, if what is being put in is a lot of crap…..

As an aside, it may also mean that the Good Stuff in future will be found behind Times style paywalls and what you get for free is increasingly Ad-peddled and raddled sh*t (as the Economist implies today – but the article is, newly, behind a paywall…..).

Facebook Privacy Algorithm Updated

News reaches us today that Facebook has reacted to the global worres about privacy abuse by hiring a Beltway Lobbyist – Washington Post:

Facebook said Thursday that it is expanding its global policy team and poached from the White House, hiring as its new vice president of global public policy, Marne Levine, chief of staff of the National Economic Council.

In her new role for the Silicon Valley social networking Goliath, Levine will oversee the company’s interaction with governments and non-governmental organizations around the globe as the company reaches 500 million users worldwide, Facebook said in an announcement Thursday.

She will be based in Washington, just as the firm builds its local policy and lobbying team to address growing interest by lawmakers and regulators on how the social networking giant is dealing with issues such as copyright, security of children online, and privacy.

Levine will also help the firm build its policy teams in Asia, the Americas and Europe, the company said.

If you can’t beat ‘em, buy a Lobbyist (or, if you pefer this homily; when the going gets tough, the tough get lobbying). We have thus updated our Facebook Privacy Algorithm to take this new step into account (see above).

The serious (as opposed to satirical) point is that Facebook has gone from groovy startup to large lobbying corporate in remarkably short time, driven in the main by the increasing conflict between their commercial model which demands exposing ever more private user data, and the growing concerns of privacy activists, legislators and – increasingly – citizens.

This follows fast on the heels on the publication of a Facebook Fanbook (every large US privacy invading company has one….)

An unsubtle hint for Facebook…..

Following the Facebook (un)privacy furore that Facebook initiated with its F8 revelations, they have now got to that stage in the process where the Apology and the Promise of Changes is made (see our analysis of the Facebook Privacy Foxtrot here). Today it came, in the Washington Post:

We have also heard that some people don’t understand how their personal information is used and worry that it is shared in ways they don’t want. I’d like to clear that up now. Many people choose to make some of their information visible to everyone so people they know can find them on Facebook.

That’s the apology, in case you missed it – you dumb or something? And now, a restatement of the Credo:

We already offer controls to limit the visibility of that information and we intend to make them even stronger.

Here are the principles under which Facebook operates:

– You have control over how your information is shared.

– We do not share your personal information with people or services you don’t want.

– We do not give advertisers access to your personal information.

– We do not and never will sell any of your information to anyone.

– We will always keep Facebook a free service for everyone.

Cough! You Wot? The systemic abrogation of the first 3 (and some suspect 4) points is the reason so many people are up in arms, dont’cha know! Still, this sort of doublespeak is pretty bog standard Facebook practice, what is more interesting is what they will actually do about it all:

There needs to be a simpler way to control your information. In the coming weeks, we will add privacy controls that are much simpler to use. We will also give you an easy way to turn off all third-party services. We are working hard to make these changes available as soon as possible.

Ah, the fine balance between giving the users what they want (easy and powerful control of privacy) and what Facebook wants (economic advantage from passing their data on) . Still, lets see what actually gets done – that after all is the acid test.

But something we suspect we will never see from Facebook (probably until regulation enforces it) is to move the default mode to Opt In, rather than Opt Out.

So, to help that process along, I include the helpful cut-out at the top of the post…..

The Roger Mellie Social Media privacy policy decision tree – Facebook late May 2010 edition

News today that Facebook has owned up to mistakes and will shortly announce steps to put it right- Mashable:

Hey,

We’ve been listening to all the feedback and have been trying to distill it down to the key things we need to improve. I’d like to show an improved product rather than just talk about things we might do.

We’re going to be ready to start talking about some of the new things we’ve built this week. I want to make sure we get this stuff right this time.

I know we’ve made a bunch of mistakes, but my hope at the end of this is that the service ends up in a better place and that people understand that our intentions are in the right place and we respond to the feedback from the people we serve.

I hope we’ll get a chance to catch up in person sometime this week. Let me know if you have any thoughts for me before then.

Mark

Apparently this was an email sent to Robert Scoble. Now Mashable, despite being very respectful of Facebook and doing the obligatory A List shtick of opining that “the privacy dustup will eventually blow over and Facebook will continue its relentless march to win the web” did make the point that:

To Facebook’s audience, it has seemed that Facebook either doesn’t know or doesn’t care about user concerns around privacy. If Facebook does indeed share user concerns and will soon make changes (as Mark explains), a very early mea culpa and increased communication with the press may have saved Facebook from a great deal of criticism.

In short: It’s great that Facebook is looking to improve its privacy settings, but explaining these moves earlier and more publicly may have been preferable.

It’s not exactly surprising though – Facebook’s standard modus operandi is arguably to push the boundary, see if there is any resistance, if there is they initially try and tough it out, then try and persuade the market that they are right and the users are wrong (initially by itself and then via proxies), and finally take a step back if there is still resistance (usually to come back at it a while later). We are about to enter the “one step back” piece of this cycle. This led me to write the tongue in cheek Facebook Privacy Reduction Decision Tree (in the style of Roger Mellie*) above.

It will be interesting to see if the proposed modifications don’t help

As one of the Mashable comments puts it:

This isn’t rocket science. Respect your customers. Don’t spring stuff on them. Do not make “sharing” the default. Give people total control over their information and protect their privacy. Don’t tell them what they want in terms of privacy and sharing, which is rude, patronizing and insulting. Especially aimed at someone who is old enough to be your mother.

The harder you have squeezed me to share, the more content I have removed from my profile, which now is almost everything possible. At this point, I will use no commercial apps, click on no ads, “like” no businesses of any kind. I do not appreciate nor will I participate in any FB presence which has been added with out my permission to sites like CNN. And this will continue to be MY policy until you prove you can be trusted.

Now, to return to the Mashable (et al) point that this will all blow over, I think the comment above implies that there is another game that may emerge here which I have captured in the system dynamic diagram below.

The Privacy / Value Vicious Cycle

In short, Facebook is not the only actor in this game – the more privacy it erodes, the less people will put on it, so the less valuable their data is, so the more they have to erode privacy….you get the picture.

How likely is this scenario? At the moment the resistance to Facebook is mainly in Geekville and starting to permeate the more intelligent mainstream media like the NY TImes, Economist etc, but it is also starting to hit various regulatory bodies and if it gets on to the mass media, thats a real tipping point.

I’d predict ( along with Mashable, SAI, GigaOm and all the other A List Apologists), that this is not the brouhaha that will trigger the tipping point. But I don’t think Facebooks ever upward march is assured. I think it will be the next one that hits the mainstream worry – and there will be a next one, its in the DNA (and probably the business plan too – that IPO and the need to declare numbers must be focussing minds somewhat).

Who is for – and against – Facebook’s UnPrivacy Policy

Today’s defence of Facebook by Tim O’Reilly (he wot coined Web 2.0) reinforced for me a very interesting thing – to wit, those rushing most to Facebook’s defence are the A List Silicon Valley bloggers.

I had a quick look via Techmeme search at the whole Facebook / Privacy furore over the last few weeks, and who was pro and anti – the resulting impression is the 2 x 2 above – and it is a very curious thing indeed.

By and large the tech-informed Bloggerati have led the criticism of Facebook’s privacy policy. By Bloggerati I mean technology bloggers, some of whom were in this space before “Web 2.0″ was even coined, and who know it well. People like dana boyd, Stowe Boyd, Nick Carr and many others have all weighed in here.

Most of the serious Mainstream media have also weighed in – initially reporting on it, but increasingly coming out against the Facebook policies. I count the New York and London Times, Economist, Guardian, BBC et al as examples in this camp. (In fact its getting to the point that the “mass” mainstream media is starting to weigh in now)

Defending Facebook – well you’d expect the magazines such as Fortune to do so, their schtick is CEO and wealth worship and they tend to believe the ends justify the means. The FT and WSJ seemed to have started off in this camp – but seem in more recent days to have moved towards the other MSM as it becomes clearer what Facebook are up to.

Which leaves us with the curious spectacle of the Silicon Valley A List Blogs – TechCruch, Read Write Web, GigaOm and now O’Reilly – all have taken their turn defending Facebook, telling us that (in effect) privacy is dead, get used to it. It’s curious in that – like the Bloggerati – they know what is going on (or not, in RWW’s case ) and once upon a time supported all the good things Web 2.0 and its associated memeplex was about.

Now, however, they twist and turn on pinhead details, telling us that Facebook is doing the right thing for us all – when just about every other technologist, ethicist, moralist, liberalist, conservative, legislator, libertarian and librarian is getting increasingly concerned.

Why are they taking this stance? To what end?, is the question that we would ask. We are not certain, but analysis of past similar events leads to a number of potential reasons:

(i) Contrarianism as an intellectual model – and it also no doubt helps drive traffic (and comments – most of their articles are full of people furiously disagreeing with them). But its interesting that they have all chosen contrarianism right now.

(ii) Same Ideology/GroupThink – this is what they genuinely believe – except in many cases their new beliefs are somewhat Pauline – the Conversion seems quite recent in many cases (in a few cases junior staff were slagging Facebook off just before the Big Guns weighed in)!

(iii) Fear of the Subject’s Wrath – this can be for overt reasons (criticising Adolf Hitler in 1930’s Germany was a life-shortening event), more subtle (You’ll never work in this town – or Valley – again) and even diabolical (You don’t get invited to the best parties anymore)

(iv) Love of the Subject’s Wealth – It has not been unkown in the past for people to radically alter their views on receipt of largesse from a rich sponsor.

Quite which of these factors (If any – there may be others) are driving our A List friends is unclear, but we would note that dependence on (or maybe proximity to) the Silicon Valley teat does seem to be a fairly good predictor of editorial viewpoint so far…..

Jus’ Sayin……

From Gawker:

According to an excerpt from Fortune columnist David Kirkpatrick’s The Facebook Effect. Hey, everyone’s got hobbies, right? From the excerpt published on All Facebook:

As the service’s engineers built more and more tools that could uncover such insights, Zuckerberg sometimes amused himself by conducting experiments. For instance, he concluded that by examining friend relationships and communications patterns he could determine with about 33 percent accuracy who a user was going to be in a relationship with a week from now. To deduce this he studied who was looking which profiles, who your friends were friends with, and who was newly single, among other indicators.

This kind of predictive capacity could be used for some pretty creepy targeted advertising opportunities: flower delivery, restaurant reservations, advice books, sexual products of various sorts.

Just so you know what can be done with a little light scraping. I wonder what one can tell from Twitter

Hat tip to commenter pseudorocket for the blog post title idea.