Three reports came out last week from Radar Networks, StumbleUpon, and ClipBlast offering what Search Insider blogger David Berkowitz calls, “more clues on how search and discovery are converging and diverging.”

“Web video is asking to be discovered” according to the ClipBlast! survey, which reports that for online video, traditional search techniques fall behind “discovery” methods popular on the social Web.

In the survey ClipBlast! asked 1,000 online consumers how they get to video on the Internet. 530 expressed a preference while 470 did not. Among the 530 respondents who had a definite opinion, “discovery” is the primary mode by which they find video online (28 percent), followed closely by recommendations from friends (27 percent). About 22 percent rely on search engines and roughly 10 percent get video from people they know only online – through social networks and the like. Relatively smaller percentages receive video from unsolicited email and RSS feeds, to which they have subscribed (5 percent, respectively).

StumbleUpon is a site that lets users discover, vote on, and share new sites through the use of a toolbar. The number of “stumbled upon” links has climbed steadily in the last two years. In the first quarter of 2008, the number of stumbles reached 974 million, 160% more than Q1 2007’s 375 million. StumbleUpon recently reached its five billionth stumble.

Eric Shonfeld of TechCrunch explains that while many people think of StumbleUpon as a “socially powered discovery engine,” rather than a new kind of search engine, he sees the site as evidence that personal discovery and search are colliding. And StumbleUpon founder Garrett Camp agrees:

Personalized search is just getting started. I think personalized crawling will start too. Crawlers now are trying to create the biggest map of the web, but implicit filtering and intelligent agents—that is where search and discovery will meet.

Nova Spivack, CEO and founder of Radar Networks, says that as we move from Web 2.0 (2000-2010) to Web 3.0 and beyond and the volume of data keeps climbing, the productivity of search will decline.

But with respect to video, these surveys and predictions may be off the mark because by and large video search isn’t very good yet. Given the paucity of robust, reliable video search online, it is hard to conclusively argue that because users aren’t searching for video in great volumes yet, they don’t want to discover videos via search.

Search and discovery will inevitably evolve as the volume of content increases online. With online video, there will necessarily be a strong connection between search and discovery; publishers will use automated discovery in an attempt to hold consumer attention, while users will want the control and specificity of search for navigating between and within online videos. The true evolution will be a user experience that allows the seamless transition between search and discovery—but until video search consistently provides the same level and quality of results as text-based search, users will continue to “prefer” discovering video because frankly, they don’t really have a choice.

In a recent study at Penn State, results showed that about 80-percent of online searches are informational in nature, whereas 10-percent are navigational and another 10-percent are transactional. Although millions of people use Web search engines, Penn State researchers show that most queries submitted can be classified into these three categories:

• Informational searching involves looking for a specific fact or topic
• Navigational searching seeks to locate a specific web site
• Transactional searching looks for information related to buying a particular product or service

This research was the first published work of its kind done using actual searching data, with the aim of real-time classification. “Other results have classified comparatively much smaller sets of queries, usually manually,” said Jim Jansen, the professor in charge of the study.

This research aimed to classify queries automatically. Our findings have broad implications for search engines and e-commerce if they can classify the user intent of queries in real time.

The study explains that nearly 70 percent of searchers use search engines as their point of entry on the Web. As such, the major search engines receive millions of queries per day and in response to those queries, present billions of results per week. The researchers asked, “What task, need or goal are these people trying to address with their Web searching?” The answer seemed to be that people are looking for information and entertainment and the majority are searching by topic. But are they finding all types of content?

One of the big challenges for media companies right now is ensuring that their content is both discoverable and consumable across the Internet. Information typically resides within documents, which is why articles and other text-based content do so well in the search economy. Similarly, with videos the core information is inside of the clips; however the only text accessible to the search engines crawlers is in the video titles, which tend to be broadly descriptive rather than specific and informative.

When a user searches CNN.com for “rising gas prices” they want to find any articles and videos that mention the topic—not just the content with their search term in the title. The process to date has relied on editors creating manual text abstracts of videos in order to get them discovered, but this doesn’t scale. Imagine if every document on the web could only be found if someone wrote an abstract! Increasingly large enterprises, especially those in the entertainment business, will need to make sure that all of their audio and video content has a consistent and complete set of meta data to unlock and exploit the types of distribution, syndication, and advertising opportunities that are rapidly emerging on the Web.

The Penn State study and resulting paper “Determining the informational, navigational and transactional intent of Web queries” drive home the importance of SEO—for all content types—and illustrate that the majority of Internet users are searching for topic-specific, newsworthy “infotainment” content. Search engines rely on rich meta data for content discovery, presentation, contextualization, and targeting advertising.

Online video usage has grown tremendously over the past years and is a trend that will continue, with 107.7 million video viewers in 2006 and an estimated 150 million by 2010 (eMarketer, February 2007). This growth has created a unique opportunity for media companies and other infotainment sites to leverage their vast amount of online multimedia assets; content producers will only realize this opportunity when they successfully optimize all content forms and connect this content to the search engines and in turn, to consumers.

In a release that went out this Monday, iProspect revealed that according to a recent study comparing universal and vertical search:

…in the case of news, image, and video results, search engine users click specialized content within general search results more than they do within vertical search results.

The study, conducted by Jupiter Research demonstrates the natural user preference towards universal search as well as the increasing importance of getting content onto the coveted first page of search results.

Only 17 percent of search engines user click a “news” result after conducting a news-specific search whereas 36 percent click “news” results within blended search results. With video, 17 percent of search engine users click “video” results within blended search results, compared to only 10 percent who click a “video” result after conducting a video-specific search. Among the various content types now showing up in blended search, “news” results were found to be the most clicked form of vertical content. And what about video; why are the numbers so low? The study noted:

Google—the largest search engine in terms of searches performed—does not offer a vertical specific search for videos on its main search page. Instead, video search is an additional click away…This is one of several probable reasons why video finishes a distant third behind images and news in terms of vertical search usage.

According to the iProspect study, it is paramount for marketers to optimize all of their digital content types so that they may be found within blended search results. The study emphasizes how important it has become for publishers to have all of their digital content turn up at the top of search engine results lists, both from a traffic-generating perspective as well as a branding perspective. Those organizations with a diverse portfolio of digital assets are best positioned to capitalize on the benefits provided by blended or universal search.

In order to successfully get these digital assets to the top of Google—and other search engines—Web publishers will need to optimize their content. One constant throughout the evolution of the Web has been the importance of text in driving search and navigation. The key to optimization across content types is the text associated with each article, image, sound bite or video clip. The ability to attribute these objects with text in the form of tags, categories and transcripts is critical to plugging this content into the top search results pages across the Web.

“Blended search allows marketers to capitalize on their digital assets without the need to affect a change in user behavior,” said Robert Murray, President, iProspect.

It essentially brings a variety of content types to users - where they are most comfortable and open to receiving it - and allows them to choose between the various result types…The bottom line is that companies that have optimized a variety of digital assets will have a distinct advantage. Those who lack such assets will essentially forfeit page real estate to their competitors.

Information Week had an interesting article last week responding to the addition of the extra site-constrained search boxes that now appear on Google search results pages.

Google believes these new destination search boxes will help make information more accessible to users as they allow searchers to conduct follow-up searches, drilling down into a specific site’s content without leaving google.com. Some find the change irksome, as searches conducted through the new search boxes mean more ad revenue for Google. As Google describes it:

Our goal is to provide the best user experience, and ads that are related to searches from competing providers are useful to consumers.

According to Google, they developed this feature to improve the user search experience—something all content producers and distributors should also take very seriously. Satisfied users stay on your site longer, consuming more content and in turn increasing your advertising revenue.

This “search within search” makes a strong case for the growing importance of good SEO practices and improved universal site search. Because the currency in the search economy is text, search engine-friendly Web pages must use text to increase the discoverability and placement of all content—including audio and video—across the major search engines. If your content is optimized so that users find specific, topically relevant pages in Google search results—and are not merely directed to your home page—odds are better that the searcher will click through to your content sooner.

But getting users to your site is only the beginning. Even IDC search analyst and Google critic Sue Feldman acknowledges that many small sites (and I would say many medium and large sites as well) have poor search capabilities—which may make users leave the site more quickly. If publishers improve their internal site search by surfacing results across multiple forms of content, they will keep users engaged; I will stay on the site longer if I can read an article, then move onto a related video clip or sound bite. For this kind of comprehensive, universal site search, publishers need to use the text currency as well. In this way they create a microcosm of the Internet’s search economy within their own site across their own content.

It may not feel fair, but it makes a whole lot of sense that Google would try to capitalize on this opportunity to both improve the user search experience online and increase their ad revenue. The best response from content producers would be to make sure their own content—text, audio, video, and image—is discoverable and consumable both through search engines and within their own sites. Google has already taken off with text content, but video is still a relatively new frontier. As such, online video offers a new opportunity for content owners to get ahead; with the right tools, content owners can take control of the optimization, distribution and consumption of their video content in a way they never could with text content. Control is essential for content owners because only by managing their brand, the context within which their content is consumed and the associated advertising can they capitalize on the growing revenue opportunities online.

According to Peter Morville, we are “at the crossroads of ubiquitous computing and the Internet” however as he sees it, “the user experience is out of control,” and “findability” will become the real story moving forward. In a recent post on Read Write Web, Richard MacManus examines Morville’s ideas, particularily his notion of ambient findability—which Melville defines as “the quality of being locatable or navigable.” Ambient findability, Morville contends, becomes more and more fundamental as information overload increases and mobile devices play a greater role in our day-to-day activities.

Morville’s most recent book, appropriately titled Ambient Findability, explores his theories on user experience and information overload. The central thesis is that information literacy, information architecture, and usability are critical components of the new world order. He further states that only by planning and designing the best possible software, devices, and Internet, will we be able to maintain this connectivity in the future.

Search is among our most important and complex challenges. As the choice of first resort for many users and tasks, search is a defining element of the user experience. And, as a unique amalgam of content, metadata, technology and design, the search results interface demands intense cross-disciplinary collaboration.

Morville believes that the future will be about something beyond search—and that something is his “findability.” His conclusion is a broad call for greater innovation—the seeds of which he sees in both Google Book Search and EveryZing. As the wealth of information continues to expand and the line blurs between on and offline activity, search becomes increasingly complicated and multidimensional. Business intelligence systems, such as software built to find and sort based on patterns, will need to bring together taxonomies and tags so that browsing and search complement one another and enhance the user experience.

Search is not broken, however the search results leave much to be desired. Meta Data, which is really just “information about information”, will be one of the biggest growth areas this decade in terms of R&D and application development. The Semantic Web vision will only happen when content producer invest in technology and systems to automatically and consistently generate accurate meta-data for their content. This meta data becomes the key to “Ambient Findability” for all content across all formats and all devices.

As globalization increases, technologies grow more advanced and human knowledge expands, we must adapt on an individual as well as a societal level. Our knowledge continues to mushroom, becoming too great to store in our print libraries and, for each person, in our minds alone. As online networks and social sites become deeply integrated in people’s lives, the virtual becomes inextricable from our “offline” activities, communities, and lives. With greater amounts of personal information moving online, we need ever more powerful and accurate search to keep our contacts, photos, videos etc. accessible. Universal search will become invaluable as people take advantage of the free storage capacity the web has to offer for everything from pictures to videos to calendars and contact lists; as our lives and our memories move online, our need for comprehensive universal search will grow exponentially. Web storage—unlike our attics and basements—must be easily navigable.

Storage Capacity

There are a number of factors that point towards the exciting direction of information storage, particularly as it relates to people’s sense of their real and virtual selves and the steady merger of these concepts into one cohesive identity. As Bhavin Turakhia, Founder & CEO of Directi highlights on his blog, Web 2.0 applications act as extensions of our desktops, storage costs are continually dropping, and people now expect nearly unlimited storage online.

The Implications

Dropping storage costs and free space to store data online—particularly within social networking sites—mean that more and more you can store anything and everything about your life at little to no cost. YouTube, Flickr, Facebook et all are already accumulating multiple facets of people’s lives online. The virtual world has grown so that what we do and how we represent ourselves online are now essential components of our identities. A recent report from accustream entitled, “User Generated Video 2005 - 2008: Mania Meets Mainstream” shows that the market for user generated videos grew by an estimated 70% in 2007, up from a total 13.2 billion views generated in 2006. These are videos average people are making and uploading; these are vignettes, pieces of our lives on YouTube. The same report forecasts that the market will continue to grow by 52% in 2008, reaching 34 billion views.

Social networking sites only want to encourage these trends. In a MediaPost article about the coming launch of the MySpace Developer Platform (an initiative in direct response to the success that rival Facebook has had with its own open developer program), MySpace COO Amit Kapur states: “This is a critical year in the evolution of the Internet” describing his focus for MySpace as creating an increasingly “personal, portable, and collaborative Web.”

The Transactive Web

And what about our own personal storage capacity: our memory? MediaPost’s Search Insider blog had an interesting couple of entries by Gord Hotchkiss recently relating to what he calls “transactive memory” and its place in the digital age. As he illustrates, we have different methods for storing our memories. Hotchkiss explains that as some people are better at remembering certain types of things, we have adapted to extend our memory capabilities collectively by using transactive memory. This neurological plasticity allows our brain to prune itself, getting rid of capacities we no longer need while strengthening those that we do.

Hotchkiss raises the questions: What about computers, and, by extension, the Internet? What about search? New technologies let us dump the details of our life on a hard drive or website somewhere and search for it when we need it. In the place of all this memory digital storage is freeing up, Hotchkiss thinks we may develop greater skills in navigating online spaces. We may improve our navigation skills, but more importantly, we will expect comprehensive, powerful universal search technology to make finding all things virtual as easy as a simple click of your mouse.

Happy New Year!

Undeniably, 2007 was an exciting year for digital media. As we enter 2008, we continue to witness the explosive growth of the next phase of the Internet’s lifecycle – multimedia.

In 2007 YouTube—then later in the year Google by association—stood as the archetype for a new age of user-generated content and viral videos. The number of people engaging with video online soared and virtual communities of every shape and size attracted users and investors as social networking established prominence as an online activity. Presidential candidates, their supporters and detractors jumped on this trend having realized the potential of such networks and social sites to spread their messages, videos, and sway voters—particularly younger constituents. As multimedia content flooded the Internet so too did the advertising dollars, pushing Newspapers and other traditional media to reexamine and reshape their own online presence.

At the end of 2007, users spent forty-five percent of their time online consuming content, and 77 percent of unique users consumed online video.

The revenue opportunities for multimedia online are currently red hot—video advertising is predicted to increase from $700 million in 2007 to $4 billion by 2011, according to eMarketer—as companies continue to look for powerful ways to impact the growing online population. Thus far however, major media companies have struggled to turn their video assets into large revenue streams online. The online “video value chain” will continue to take shape in 2008 as more companies bring creative solutions to questions of content production, delivery, discovery and consumption. This year will bring even greater innovation to the Internet video space as content producers and consumers negotiate the evolving sphere of online multimedia.

Andy Plesser of Beet.TV recently spoke with EveryZing CEO Tom Wilde about video search. Tom explains that universal search creates the need to better unlock the contents of multimedia files online to deliver a complete user experience. Check out the interview at Beet.TV.

A number of articles and blogs have noted AOL’s recent launch of Truveo. The new destination site proclaims, “Truveo’s mission is to improve the quality of video search so that you can always find the right video to watch.” Why launch a new, destination site? Truveo founder and CEO Tim Tuttle contends,

“Video on the Web is no longer about user-generated content on sites like YouTube. There are now a lot of quality sites with other videos.”

Tuttle explains that Truveo is trying to bring that quality content to consumers.

Truveo was founded in 2004 to power search on other web sites. AOL purchased the search engine in 2006. The decision to launch Truveo.com outside of the AOL brand has people wondering about AOL’s goals and strategy for the site.

Much like Google, Truveo utilizes the metadata text linked with video clips to search across the Web. Truveo distinguishes itself with its use of multiple searching techniques, including a “page inspection” that looks at the Web page by page to identify videos.

While the site is well designed, it still seems to be an incremental improvement rather than revolutionary, as Truveo still indexes essentially only titles and tags. The analogy would be to imagine someone building a web search engine today that indexed only document titles and not the page content. The other peculiar part of AOL’s strategy here is the fact that the rest of the search market is moving towards a universal search experience rather than a vertical search experience, mainly because users don’t do vertical search in any large numbers. They expect the “web search” box to deliver them all the relevant content, regardless of format, in response to their query. Nonetheless the online video market continues to evolve at a rapid pace.

This entry is a follow up to the previous one I wrote last week titled: What’s Worth More: Online Eyeballs or Digital Media Content written right about the time the drums of war between Google and Viacom were being heard ’round the world. As the whole world knows by now, Google finally got sued over its subsidiary’s (YouTube) claimed copyright infringement of Viacom’s content. Many experts in the field had anticipated that YouTube would eventually get sued (including Google…the company set aside a couple of hundred million dollars as part of the YouTube acquisition to fight copyright infringement litigation) for allowing its users to upload video content on the site, with little to no screening of the content to make sure such content truly belongs to the user uploading “the stuff”.

What’s a kicker to me is that both Google and Viacom were rumored to be in serious negotiations to put Viacom’s content on the YouTube site, in exchange for some type of revenue guarantee by Google, just before it fell apart last week. Google is known in the search industry for paying top dollar to partners to become their exclusive provider of Web Search solutions (that includes web search and sponsored listings), see Google press release Time Warner’s AOL and Google to Expand Strategic Alliance. It’s not hard to imagine that Viacom tried to arm wrestle Google and get them to agree to some outrageous revenue guarantees in exchange for Viacom’s content.

If that’s how the negotiations between these two companies happened, well I don’t think Google was ready to open its wallet and sign that big fat check as they’ve done in the past. In the absence of a negotiated agreement, often times companies resort to suing other companies claiming all sorts of outrageous violations to see if one sticks in front of a judge. In the case of Viacom, I find it hard to believe a judge is going to buy their claim. Viacom’s lawyers and executive team rolled the diced to go for the big pay check. In the end, when all this mess is settled, I think they’re going to wish they had taken Google’s offer, whatever that was.