social network effect

By passing this meme on through the blogosphere hopefully we can generate more awareness and avert a serious tragedy. As concerned world-citizens this something we bloggers can do to help.

How to participate:

1. Copy this entire post to your blog, including this special number: 1081081081234

2. After a few days, you can search Google for the number 1081081081234 to find all blogs that are participating in this protest and petition. Note: Google indexes blogs at different rates, so it could take longer for your blog to show up in the results.

3. If you know how to add tags to your blog posts, add the Technorati tag 1081081081234 to your post as well. This will make your post findable sooner in Technorati.

THE SITUATION IN BURMA AND WHY IT MATTERS TO ALL OF US

There is no press freedom in Burma and the government has started turning off the Internet and other means of communication, so it is difficult to get news out. Individuals on the ground have been sending their day-by-day reports to the BBC, and they are heartbreaking. I encourage you to read these accounts to see for yourself what is really going on in Burma. Please include this link in your own blog post.

The situation in Burma is increasingly dangerous. Hundreds of thousands of unarmed peaceful protesters, including monks and nuns, are risking their lives to march for democracy against an unpopular but well-armed military dictatorship that will stop at nothing to continue its repressive rule. While the generals in power and their families are literally dripping in gold and diamonds, the people of Burma are impoverished, deprived of basic human rights, cut off from the rest of the world, and increasingly under threat of violence.

This week the people of Burma have risen up collectively in the largest public demonstrations against the ruling Junta in decades. It’s an amazing show of bravery, decency, and democracy in action. But although these protests are peaceful, the military rulers are starting to crack down with violence. Already there have been at least several reported deaths, and hundreds of critical injuries from soldiers beating unarmed civilians to the point of death.

The actual fatalities and injuries are probably far worse, but the only news we have is coming from individuals who are sneaking reports past the authorities. Unfortunately it looks like a large-scale blood-bath may ensue — and the victims will be mostly women, children, the elderly and unarmed monks and nuns.

Contrary to what the Burmese, Chinese and Russian governments have stated, this is not merely a local internal political issue, it is an issue of global importance and it affects the global community. As concerned citizens, we cannot allow any government anywhere in the world to use its military to attack and kill peacefully demonstrating, unarmed citizens.

In this modern day and age violence against unarmed civilians is unacceptable and if it is allowed to happen, without serious consequences for the perpetrators, it creates a precedent for it to happen again somewhere else. If we want a more peaceful world, it is up to each of us to make a personal stand on these fundamental issues whenever they arise.

Please join me in calling on the Burmese government to negotiate peacefully with its citizens, and on China to intervene to prevent further violence. And please help to raise awareness of the developing situation in Burma so that hopefully we can avert a large-scale human disaster there.

• Make Your Own Shark Video
• The Devolution of Social Networking
• Fun (But Somewhat Obscure) Toys At Digg
• Opportunity Is Knocking

ShiftHappens. If you haven’t watched it yet, go now.

The world is going to be a very different place very soon. Think our social organizations will change with it?

[Congratulations to jbrenman on winning The Worlds Best Presentation Contest with his yummy adaptation of Karl Fisch’s very-viral ShiftHappens talk.]

Update:  Mark Morford, riding the cosmic karma train as usual, has a nice bubbly read about the Shift & the Singularity.

From the “Is Web 2.0 a Bubble?” files comes this tidbit: Social networking has gone to the dogs.

No, literally. SNIF labs, a spinout of MIT’s Media Lab, has developed a wireless-enabled dog collar that records and tracks all of the “social interactions” your pooch shares throughout the day. When you get home from walksies, this data is automatically uploaded into a central web site, allowing you to see who Jake’s “best friends” are, and connect with their owners.

My first thought: the apocalypse is upon us. Over-hyped online pet services were the poster child for the first dot-com collapse. And SNIF is not the only company playing in this space. Web 2.0 is doomed.

My second thought: how silly. Dogs do not need a web browser to know who their friends are. But then again, people shouldn’t either, and that hasn’t stopped MySpace’s meteoric rise to $ucce$$.

My third thought: what a damn fine way to get a date with that hottie with the black lab at Duboce Park. But then I remembered that I’m married, and I don’t have a dog.

Still, that last thought let me see the why SNIF is not stupid, or silly. It’s really quite smart.

The key to SNIF is that it is one of a handful of services that bring social networking into the physical world — but with a critical difference. When Dodgeball tells you that FOAFs are within a 10-block radius or Jambo tells you that your buddies are on your local wireless network, they are leveraging your virtual social network to help create real-world linkups. SNIF is unique in doing the reverse: it documents real-world interactions to create virtual social networks.

While targeting the doggie set for initial rollout is probably a smart marketing move, there’s no reason that SNIF’s core technology can’t be leveraged in other areas. Who the hell says that you need to put that little wireless device on your dog? How about putting it on people at your hot tech rollout party? Or at your amusement park? Lab equipment? Herds of migrating wildebeest? Ravers at Burning Man? The core technology could be used anywhere you want to track people, animals or objects bumping into each other in order to understand their interactions (within cost limitations, of course). I’m sure the bright folks over at the Media Lab have a wiki filled with potential applications …

I have just one piece of advice for the founders: move the beta test from Boston to SF. In a city where there are more dogs than children and web entrepreneurs are falling out of trees, you’ve got two built-in communities sitting up and begging to be networked.

Never trust a programmer who says, “I can give you 90% of what you’re looking for.”

I say this from hard-learned experience. When it comes to data, 90% is worse than nothing. Since a user can never know where that missing 10% is, they end up mistrusting everything. You’ve shoved them into the realm of Rumsfeld’s “known unknowns” and the half-truths of Alberto Gonzales’ congressional testimony, and it drives them crazy.

I should know. I recently led a project where the stated goal was accurate access to 90% of the data that was locked up in an otherwise unqueriable database. Once we rolled out at 90%, though, we spent the next six months dogged by our users to get the damn thing as close to 100% as we possibly could. They wouldn’t settle for anything less.

It’s a good war story — you can check out the details here, if you’re interested. But in this post I’d rather focus on what this dynamic means for Semantic Web technologies — DBpedia in particular.

While I was playing with the various query interfaces into DBpedia over the past few days, I had a sinking sense of deja vu. It’s not that the queries didn’t return useful information, but I was always left with the sense that I wasn’t getting the complete picture. Why are the doctral advisors for only 69 scientists available? Haven’t we flown more than 11 space missions since Vostok-1 and Mercury-3 were launched? Is this really a complete list of all academy award winners? Isn’t the California Golden Bear an exinct mammal? And so on and so forth. Even though the data that was returned seemed accurate, it was impossible to tell what I was missing. It was frustrating.

Now when I rolled out a data warehouse with 90%, the users had only one recourse if they wanted more — they had to hassle me, the administrator, to make it show up in the system. That was so old school I almost felt I should be doing the data entry with punch cards. It was the exact opposite of the network effect: every request needed to be routed through a central chokepoint. Call it the bottleneck effect. It was a recipe for frustration for everyone.

But hey, we’re talking Web 2.0! We should be leveraging the social network effect to the hilt. A central adminstrator shouldn’t be required to update inaccuracies or omissions in the data store — we should be allowing users to do it themselves! They’re certainly going to be motivated enough, especially if they’re going to be trolling through the same data again and again. Give the users the ability to correct any inaccuracies and omissions they may find and you’ve flipped the problem on its head. The missing data isn’t frustrating; it’s an invitation to contribute!

Unfortunately, DBpedia resembles my old-school data warehouse much more than the freely-editable encyclopedia from which it takes its inspiration. Everything that’s available has been distilled from millions of wiki pages and shoved into a far-off RDF datastore with no data entry capabilities. To add “Golden Bears” to the list of extinct mammals, I would need to search wikipedia for the correct page, scan the infobox for the status column, update it to EX, and hope that some extract routine would pick it up the next time it ran.

The social network effect has been short-circuted. The activation energy required to get that simple update in there is just too high — too much work for too little benefit. The data store has become more reliant on the parsing and extraction efforts of a handful of programmers than the collective efforts of the internet community.

Without this, it’s unlikely DBpedia will get past the interesting proof-of-concept phase — which, in all fairness, is probably what it was created to be. But it’s enough to get me thinking of what could be, if only there was some magic that allowed the easy data entry need to grow that 90% into something amazing …

A beautiful link dropped into my inbox last weekend, courtesy of Stewart Brand and the Long Now foundation:

It began on a New Jersey beach. Frans Lanting was photographing horseshoe crabs for a story about how they are being ground up for eel bait and at the same time their blood is used for drug testing—a $100 million industry. The crabs have primordial eyesight, which they employ mainly for finding sex partners. Photographing the horseshoes having a spawning orgy one spooky twilight, Lanting felt like he was suddenly back in the Silurian, 430 million years ago…

(c) Frans Lanting

So Lanting and his wife Chris Eckstrom set out in search of “time capsules,” places on the present Earth where he could find and photograph all the ancient stages of life. A two-year project expanded to seven years …

Lanting has put together a web-based slideshow of his presentation, a gorgeous romp from the Precambrian to the Pleocene. It’s well worth your visit: http://www.lifethroughtime.com/experience.html.

(As a side note, the Long Now Foundation is now accepting charter memberships. Donations are tax-deductible and include access to the foundation’s video seminar library, a virtual treasure trove of presentations from the likes of Jimmy Wales, Philip Rosedale, Vernor Vinge, Phillip Tetlock and others. Only a thousand will ever be made available, and spot #635 was just snagged by yours truly … join today!)

“It will never work.”

I have to admit it, that was my first thought when I first stumbled across the Semantic Web.

Not for any technical reason, mind you. While I’m sure there are technical challenges gallore to be found in implementing the Semantic Web, none seem insurmountable. No, my issue was more fundamental. RDF is a structured knowledge representation, and most people hate structuring their knowledge for a computer’s consumption. All of this RDF and tuples and ontologies may be a cog-sci student’s wet dream, but let’s be frank: it leaves most of our users out there — the actual user communities we’re trying to ensnare, mind you — stone cold.

This is a huge blind spot for most computer scientists, because we are the rare mutants who actually like ordering our thought processes into a logical and structured format. (That’s what programming and data modelling is, isn’t it?) But remember: most of the world still sees us for the geeks we are, and they don’t really like the way we think. They like their thoughts and knowledge to remain fuzzy and abstract. They just think their thoughts and know what they know, and they don’t want to be bothered with elucidating their knowledge in a structured format. If they want to record or transfer their knowledge, most people prefer to use the traditional method our wetware has used to sync up for millenia: natural language.

You know, that data representation that computers really suck at understanding?

Sure, once knowledge is in RDF, you could theoretically start doing all sorts of amazing things with it … but where exactly was that RDF going to come from? Who was going to take the time to encode their knowledge in a standardized format? Why on earth would they waste their time? Nevermind the issues with undertsanding ontologies or the lack of data entry tools. Even given the perfect tool, getting people to enocode useful information into RDF would be like getting Dick Cheney to write buddhist poetry. It just wasn’t going to happen, not in this lifetime, and certainly not on a wide enough scale to really get the network effect primed.

Or so I thought — but then I stumbled upon dbpedia. And lo, my eyes were opened unto the light.

You see, the clever folks over at the University of Leipzig and the Freie Universitat of Berlin had a wonderful realization. There’s a huge number of people who are already actively uploading their knowledge into that miracle of collective effort and social networking known as wikipedia. Might not there be some nuggets of RDF-parasable information in that huge mass of natural-language-based knowledge?

And indeed there are. In the upper-right corner of many wikipedia pages is an “infobox“, a standard template used to give quick facts and summary information about the current entry. Different categories of entries have different infoboxes. For instance, there’s an infobox for countries, and one for movies, and albums. Each category’s infobox is structured the same, and wiki authors have by collective agreement started to manually enter key information in this structured format.

There you have it. Real live human beings, linked via wikipedia into the network effect, entering data in a structured form. A structured format that just happens to look a whole lot like RDF, by the way. So why not get busy and parse it into an RDF datastore?

The result is dbpedia, 91 million tuples of user-generated RDF goodness, with a few nifty web query interfaces allowing access to it all. Later this weekend we’ll explore what you can do with this resource.

So maybe there is something to this semantic web thing, after all.

(Of course, freebase may have solved this problem in spades, but I won’t know until I learn the secret handshake required to get an alpha account …)

The concept of the sematic web is a few years old now, but is only now really beginning to gain real-world traction. The idea is based upon the simple observation that the current web mainly consists of a network of human-readable documents, not computer-parsable data. Because of this, the web is extremely useful for humans to gather data and information, but not at all useful for computers. The sematic web seeks to overcome this limitation by promoting standards for information representation and exchange to create a “web of data”. Much in the same way that technical interchange standards like HTTP and HTML allowed the organic growth of the “web of documents”, new technical standards will provide a fertile ground for the growth of this “web of data”.

The key technical standards for the Semantic Web are RDF and OWL, both of which were concieved by Tim Berners-Lee and later developed into working standards by the collective efforts of many contributors to W3C working groups. These standards provide a consistent, unifed way of representing knowledge and information as well as mechnisms for exchanging this information. There is also SPARQL, the emerging standard query language for RDF data stores.

So what can you do with the Semantic Web? Theoretically, lots. Consider the following simple question: what are the homepages for all of the Web 2.0 companies located in San Francisco? With today’s tools, this is a nearly impossible question to answer. Typing “web 2.0 company san francisco” into Google returns a confusing mishmash of 12 million hits, most of which are neither companies nor located in San Francisco. It’s up to you, the human on the other side of the screen, to sift through the dross of ads, conference announcements, articles, and blog chatter to find the few gems you are looking for. It’s also up to you, the human, to cut/paste all of these into a spreadsheet for tracking.

This is a royal pain in the ass. I should know — I’ve tried to compile this list, and was quickly frustrated.

The Semantic Web solves this problem by providing a standard mechanisms for web sites to publish data, instead of documents. One could imagine that every company that wanted to make its presence known on the Semantic Web would publish a set of RDF tags (<MyCompany, location, San Francisco> and <MyCompany, field, Web 2.0>) describing itself. With the information in a standard format, query tools could then allow construction of targeted queries that answer the specific question at hand.

Of course, this is only a basic example. Imagine if scientific publications provided RDF representations of the data contained within them, or if data repositories like NCBI and PubChem provided RDF gateways into their data. We’ll be exploring these questions in an upcoming entry.

I also think there’s a lot of promise in using RDF data stores as a simple replacement for standard relational databases, especially in environments that require very dynamic data models. Since RDF entries in essence define their own schema, developers are no longer tied to a fixed data model. As the domain space changes and the data representation evolve, RDF can transparently allow entry of new attributes for key data. Anyone who has tried to maintain a LIMS will quickly understand the power of this method.

Radar Networks is one of a handful of companies that are focused on bringing “sematic web” technologies to market. Other players in the field include the startups Metaweb (developers of Freebase), Zephira, and Franz. There are also a variety of public/aceademic/opensource efforts, including SIMILE, Jena, and dbepdia. But there are heavyweights, too — players like Oracle are active in the space.

The field is just emerging from its infancy, but many see a bright future ahead for the Semantic Web. We’ve seen what the Network Effect can do for document repositories, software development projects, and community building. Just imagine what it can do with the world’s collective knowledge!

OK, let’s start with the basics: the fundamental dynamics of the network effect. All of you who have worked worked in tech anytime in the last thirty years can skip to the next post, since this will be as obvious as gravity to you. But if you’re new here, read on.

The classic example of a network effect is the fax machine. Consider, for a moment, that it’s 1974 again and fax machines are the next whiz-bang technology ready to hit the marketplace (instead of a creaking technology rapidly moving into obsolescence). Nobody really has one of the things yet, and most people don’t even know what they are. It’s a revolutionary techonology, to be sure — sending pictures over a phone line, imagine that! — but you, as a business owner, aren’t really sure that it’s time to buy one yet.

What is the value of having the only fax machine in the world? Absolutely zero — who could you fax anything to?

Once there are two fax machines, however, there is some real value. You can send copies of documents from point A to point B. Of course, that’s still a pretty limited use case. Perhaps if you had a single remote office, you could send copies back and forth between the two sites. Or you could set up collection and routing facilities around each of the fax machines, and create a sort of “rapid mail system”.

Each of these machines would be extremely expensive, of course. Being the only two machines of their type in the world, all of the R&D cost required to create this fancy new technology would have to be amortized over these two machines. Only very deep pockets — e.g., big corporations and governments — could afford them, and they would naturally be stingy with their use. Not very useful: let’s say this scenario has value=1, for the one connection that exists.

Now, imagine there are a hundred fax machines in the world. They’d still be expensive as all hell, but they are far more useful than when there were only two available. Different offices could fax documents to each other without needing to go through complicated land-based routing on each side. The single connection possible when two machines existed has been replaced by 10,000 (ie, 100^2) connections, so we’ll say the value of this scenario is 10,000.

As a technology becomes yet more widespread, the value begins to skyrocket. Imagine the capabilities available when 10,000 fax machines are in the world. Now most large-sized companies have at least one, so companies can use them to communicate with each other. When there are 100,000 fax machines in the world, it becomes expected that any two parties will be able to find a fax machine to communicate with. When there are a million fax machines, people begin to rely on it for their everyday work. There are now ONE TRILLION possible connections in the network.

Lest we forget, as the value of these devices has been skyrocketing, their cost has dropped. By the time there are a million fax machines in the world, the technology has become a mass-market commodity, available to every home or small business that wants one.

Interestingly enough, this scenario was first described by one of the inventors of the Internet, Robert Metcalf, and is known as “Metcalf’s Law”. So it should come as no surprise that network effects are so fundamental to the working of the grandest of all networks — the Internet itself.

To summarize:

Rule #1: In a networked environment, the value of a good or service goes up by the square of the number of participants in the network.

Rule #2: The cost of a networked good or service goes down as the number of participants goes up, since the cost of developing and manufacturing the good is amortized over more buyers.

Welcome!

By some twist of fate, some random working of karma, you have been drawn to Social Network Effect. Welcome! Perhaps it was just a link gone wrong, or a hiccup on Google’s servers. Or perhaps it was a more directed search that led you here — the emailed link, the whispered reference, the workplace directive. No matter. All that matters is that you are here now, and that you will be here again. Welcome. We hope you enjoy your vists.

What is Social Network Effect? Well, in the spirit of “Web 2.0″ and the new millenium, social network effect is a mashup — a mashup of two terms, both powerful in their own right, that taken together just might have the potential to transform the world.

Social Network — A web of people, interlinked through shared ties.

Network Effect — The principle that the value of a good or service increases exponentially based upon the number of people using that service.

Thus, the social network effect: the principle that the value that individuals can provide to the world — through shared work and organized effort — will increase exponentially as our lives become increasingly interconnected through the miracle that is the Internet. The world is ready to shake off the dreary chains of hierarchy and control that have been both our bane and boon since the Egyptians first raised pyramids along the Nile. In our hearts, we’ve always known we could do better — and now we can. We’ve seen it happen already: first Linux, then Apache, now Wikipedia. Not to mention the millions of blogs, wikis and social networking sites that have bloomed through cyberspace like fertile field after a soft spring rain.  We can organize ourselves into performing productive work without the carrot and the stick, without the wary eye of the boss overlooking our efforts.

This blog is an exploration of the idea that the Internet allows new forms of social organization that are smarter, more efficient, and more fun than the ways we’ve traditionally organized our lives and our work. More than that, this blog will become an embodiment of that ideal, by creating a network of like-minded individuals who are working to bring about this shift.

Along the way, there’s sure to be lots of fun. There will be wikis and web 2.0, social networking sites and open-source software. We’ll explore Mechanical Turks and the economy of Burning Man, tag clouds and semantic networks — all with the end goal of creating new ways of organizing ourselves and our lives.