Blekko search engine with a human touch

A small Silicon Valley company with some big name backers has released a test version of a new search engine that the company says has a key ingredient that is missing from Google, the human touch.

According to Blekko, the Web has increasingly become saturated with spam-like websites, specially designed to pop up in Google's search results but whose content is heavier on marketing pitches than substantive information.

The remedy, Blekko Chief Executive Rich Skrenta told Reuters, is to narrow searches to groups of websites that people, not computers, have pre-approved as being the best sources of information for particular topics.

The approach is decidedly old-school in an industry where computer algorithms developed by engineers at Google and Microsoft Corp have for years been considered the ideal way to find information in the sea of online data.

Blekko has raised $ 24 million in funding as it has developed its product over the past three years, with angel investors including Marc Andreessen, the creator of the first Web browser, and Ron Conway, who has invested in tech companies including Twitter, Foursquare and even Google.

Blekko joins a long list of search engines that have tried to improve upon Google's business, with a less-than-stellar track record. Cuil, a high-profile search engine launched by former Google employees in 2008, quietly shut down in September; Powerset, a search engine that let people ask questions in plain English, was acquired by Microsoft in 2008.

Greg Sterling, an Internet consultant and a contributing editor for the online blog Search Engine Land, said he doesn't expect that Blekko will displace Google any time soon. But he said the company has developed a creative approach to search that might become popular as a secondary search engine for certain types of queries.

In an emailed statement, Google said that it welcomes competition that helps users get useful information and gives people new choices. "Having great competitors is a huge benefit to us and everyone in the search space-it makes us all work harder, and at the end of the day everyone benefits from that," said Google.

Blekko is launching with a special directory of websites that can provide spam-free results in seven general search categories: health, recipes, song lyrics, hotels, automobiles, colleges and personal finance.

Blekko also allows users to create their own personal directories of websites for any topic, so that the search engine only looks for information from sources the user deems relevant or trustworthy.

The focus on quality websites over quantity has appeal, said Sterling. "We don't care if there are 30 million results or 40 million results, it's really 'Give me the information I need in an efficient way so I don't have to go wading through all this nonsense'," he said.

Robots.txt

Robots.txt

It is great when search engines frequently visit your site and index your content but often there are cases when indexing parts of your online content is not what you want. For instance, if you have two versions of a page (one for viewing in the browser and one for printing), you'd rather have the printing version excluded from crawling, otherwise you risk being imposed a duplicate content penalty. Also, if you happen to have sensitive data on your site that you do not want the world to see, you will also prefer that search engines do not index these pages (although in this case the only sure way for not indexing sensitive data is to keep it offline on a separate machine). Additionally, if you want to save some bandwidth by excluding images, stylesheets and javascript from indexing, you also need a way to tell spiders to keep away from these items.

One way to tell search engines which files and folders on your Web site to avoid is with the use of the Robots metatag. But since not all search engines read metatags, the Robots matatag can simply go unnoticed. A better way to inform search engines about your will is to use a robots.txt file.

What Is Robots.txt?

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site and the fact that you put a robots.txt file is something like putting a note “Please, do not enter” on an unlocked door – e.g. you cannot prevent thieves from coming in but the good guys will not open to door and enter. That is why we say that if you have really sen sitive data, it is too naïve to rely on robots.txt to protect it from being indexed and displayed in search results.

The location of robots.txt is very important. It must be in the main directory because otherwise user agents (search engines) will not be able to find it – they do not search the whole site for a file named robots.txt. Instead, they look first in the main directory and if they don't find it there, they simply assume that this site does not have a robots.txt file and therefore they index everything they find along the way. So, if you don't put robots.txt in the right place, do not be surprised that search engines index your whole site.

The concept and structure of robots.txt has been developed more than a decade ago and if you are interested to learn more about it, visit or you can go straight to the Standard for Robot Exclusion because in this article we will deal only with the most important aspects of a robots.txt file. Next we will continue with the structure a robots.txt file.

Structure of a Robots.txt File

The structure of a robots.txt is pretty simple (and barely flexible) – it is an endless list of user agents and disallowed files and directories. Basically, the syntax is as follows:

User-agent:

Disallow:

“User-agent” are search engines' crawlers and disallow: lists the files and directories to be excluded from indexing. In addition to “user-agent:” and “disallow:” entries, you can include comment lines – just put the # sign at the beginning of the line:

# All user agents are disallowed to see the /temp directory.

User-agent: *

Disallow: /temp/

The Traps of a Robots.txt File

when you start making complicated files – i.e. you decide to allow different user agents access to different directories – problems can start, if you do not pay special attention to the traps of a robots.txt file. Common mistakes include typos and contradicting directives. Typos are misspelled user-agents, directories, missing colons after User-agent and Disallow, etc. Typos can be tricky to find but in some cases validation tools help.

The more serious problem is with logical errors. For instance:

User-agent: *

Disallow: /temp/

User-agent: Googlebot

Disallow: /images/

Disallow: /temp/

Disallow: /cgi-bin/

The above example is from a robots.txt that allows all agents to access everything on the site except the /temp directory. Up to here it is fine but later on there is another record that specifies more restrictive terms for Googlebot. When Googlebot starts reading robots.txt, it will see that all user agents (including Googlebot itself) are allowed to all folders except /temp/. This is enough for Googlebot to know, so it will not read the file to the end and will index everything except /temp/ - including /images/ and /cgi-bin/, which you think you have told it not to touch. You see, the structure of a robots.txt file is simple but still serious mistakes can be made easily.

Tools to Generate and Validate a Robots.txt File

Having in mind the simple syntax of a robots.txt file, you can always read it to see if everything is OK but it is much easier to use a validator, like this one: http://tool.motoricerca.info/robots-checker.phtml. These tools report about common mistakes like missing slashes or colons, which if not detected compromise your efforts. For instance, if you have typed:

user agent: *

Disallow: /temp/

this is wrong because there is no slash between “user” and “agent” and the syntax is incorrect.

In those cases, when you have a complex robots.txt file – i.e. you give different instructions to different user agents or you have a long list of directories and subdirectories to exclude, writing the file manually can be a real pain. But do not worry – there are tools that will generate the file for you. What is more, there are visual tools that allow to point and select which files and folders are to be excluded. But even if you do not feel like buying a graphical tool for robots.txt generation, there are online tools to assist you. For instance, the Server-Side Robots Generator offers a dropdown list of user agents and a text box for you to list the files you don't want indexed. Honestly, it is not much of a help, unless you want to set specific rules for different search engines because in any case it is up to you to type the list of directories but is more than nothing.

Importance of social media marketing

According to me Social Media Marketing is the one of the most trending topics in online marketing these days and you can see a lot of articles and publications on social media marketing, its importances, ways to implement and so on. Most of the companies are trying to get themselves engaged in various social media channels for various purposes. If planned and executed well Social Media marketing can be one of the most cheapest and effective mode of online marketing and promotion. If your social media marketing is not well planned and executed, it may end in a disaster.

Here I would like to mention 5 major importance of social media marketing for a company.

1) Branding

Companies can use social media channels as a way for increasing its goodwills and trustworthiness which may ultimately result in better branding. Companies can have conversation with its customers, which makes a great amount of credibility among its customers.

2) Lead Generation

Lead generation is another reason why companies should opt for social media marketing campaigns. Lead generation is an integral part of a company’s success and social media marketing strategies and campaigns helps in targeted and relevant lead generation.

3) Engage with its customers

Social media channels helps it more easier and flexible when it comes to engaging with its customers. Customer engagement is the most important benefits of social media marketing and this helps to retain its existing customers and also increase brand credibility. During the recent economic downfall, a lot of companies succeeded in retaining its customers with the help of customer engagement through social media channels.

4) Generate Relevant Traffic

Traffic generation is the sole purpose of all the online marketing campaigns and social media channels are highly powerful traffic generation sites. There are many niche social media channels which makes it easier to generate relevant and quality traffic to the website.

5) Improve ROI

Promotion of your products in relevant and product niche social media channels can help you in increasing your return over investment (ROI). Industry relevant social media marketing can help in making your conversion rates better and enjoy a enviable ROI.

6) Getting Customer Feedbacks

Customer feedbacks are the backbone of any industry and the feedback helps in improving the products as per the customer’s requirements. Since social media channels helps in making customer engagement much easier, customer feedbacks are always there on the top and one can always expect helpful customer reviews and feedback which can help in making the products or services even more better.

7) Announcing New products or services

There is no other better and cheap media than the social media sites to inform the customers about the arrival or release of new products and services. Social media channels helps it easier to spread the news of new product and service arrivals to targeted audience.

8) Know more About Customer Preferences

Social Media allows you to know more about the trends and preferences of the customers and act accordingly. Customer preference can also be related to the 5th and 3rd points mentioned above. Knowing the preferences of your customers makes it easier for you to enhance your product and plan your online marketing campaigns in much more effective way.

9) Influence on Search Engine Rankings

Search engines are the most relevant and consistent traffic generator for any website. Effective Social Media campaigns can bring out a remarkable influence on your search engine rankings. Since most of the search engine giants are going on with real time search results, the social media updates and bookmarks can help your site rank much better in the search engines.

10) Enhance Customer Relationship Management

Customer relationship is an integral part of the growth of any company and when customer relationship fails, the company and its operations are also bound to fail.