Social Science Statistics Blog

In developing this line of work, we were actually struck by ability of the search engines to find these meaningless documents, yet provide no indication whether they were meaningful (along the same lines, there has been attention paid to the problems of malicious placment of bogus information in Wikipedia.) But what's uncomfortable is realizing that search results are affected by this bogus data. And with the growing tacit agreement that the order of the list is somehow related to a document's worth or meaning, e.g., citations, popularity, there seems to be a significant problem here. There are also technical problems faced by groups who's main vocation is working on and through the web--most of the significant problems are computer-mediate--automation. We thought of taking a first strike by seeing whether we could distinguish between bogus scientific papers that are generated by a computer from simply human-authored texts. We were successful in this aspect. We actually devised several protocols for generating this bogus scientific texts. The application in its current form is not meant to be able to discern human-authored bogus texts from authentic ones. We have begun looking into this now. We have been asked, interestingly, to consider some of the social problems you've posed here. And we've already been made of aware of an "arm's race" as it were. A good deal of spam is utilizing this automated letter writing to fool classfiers--but as many of you likely have noticed, the sentences don't make sense. I'm happy to discuss any questions or problems we might find interesting. It's unclear why this procedure works--we are trying to formalize this structure as well. Lastly, the application performs poorly on short texts--entropy is too low likely.

Best,

Memo Dalkilic (School of Informatics/Center for Genomics and Bioinformatics [Indiana University])