Cloud analysis of texts with TagCrowd

Résumé:
Tagcrowd est un outil en ligne de mesure de la fréquence des mots dans un texte. C'est un outil pratique, gratuit, dont les paramètres sont adaptables. Il existe plus d'options pour les textes en langue anglaise. Pour un historien, parmi les nombreuses utilisations vantées par le concepteur, deux utilisations apparaissent comme séduisantes:
- pour un premier travail de débroussaillage de données textuelles longues, dans une perspective comparative.
- pour illustrer son propos par la génération automatique d'un nuage de mots-clefs visuellement efficace.
Bref, c'est un outil amusant. En voici une analyse plus poussée:

In dealing with increasingly massive amount of texts which are more and more available in more than image forms, researchers in Humanities can be tempted to get computer's help.
A very basic way to approach a text with a computer is by simply do a search with key-words, i.e. with the words I think will be inside the text or with the words I hope will be inside the text. That means doing exactly like browsing through an index at the end of a book.
But softwares can help us do just the opposite: faced with huge amounts of text, we can generate indexes on the basis of word frequency. Thus we can have our own tags generated automatically.
Computerised text analysis is a very advanced field and provides many different approaches to help researcher in what they most urgently need : devices to simplify the analysis of larger and larger amounts of text data a single human being would be unable to process in a day, a year, or a life-time. Text analysis can be based on proximity (geometrical algorithms), on frequency, on meaning, on all of the above, etc.
TagCrowd is one of these tools: it is a web application that creates clouds of tags selected on the basis of frequency in a given text.

Let's test it with two pieces on China and religion :

- La Religion des Chinois, by Marcel Granet (one of the founding fathers of modern Sinology in France), 1922:

c'est (101) chine (61) chinois (94) ciel (123) culte (125) d'un (183) d'une (178) famille (119) fils (86) fut (130) l'on (81) l'ordre (67) pere (62) point (141) principe (66) puissance (87) qu'il (93) religieux (63) religion (109) seigneur (67) sol (96) temps (98) terre (80) vertu (76) vie (168)

created at TagCrowd.com

This has been obtained while specifying the elimination of most common words in the French language. But it is not perfect since "d'un"or "d'une" were not eliminated.
But it is much better than that:

au (398) aux (252) dans (593) de (2769) des (1414) du (741) en (617) est (457) et (1395) il (553) la (2165) le (1514) les (1604) leur (272) ne (382) ou (384) par (510) plus (308) pour (356) que (637) qui (613) se (436) son (235) un (653) une (576)

created at TagCrowd.com

So to solve the problem of unwanted words or compounds, one can create a stoplist. I did that to improve the quality of the cloud, and I obtained that (while expanding the number of words to 50):

ancetres (60) chef (56) chine (61) chinois (93) choses (45) ciel (123) croyances (54) culte (125) cultes (56) d'abord (51) dieu (59) dieux (41) droit (44) famille (119) femmes (55) fetes (61) fils (86) furent (53) grand (41) groupe (54) hommes (45) l'idee (42) l'ordre (67) lettres (49) lieu (50) monde (53) mort (58) n'est (54) nature (61) nom (49) pays (48) pensee (41) pere (62) point (141) pouvoir (46) pratiques (55) principe (66) puis (43) puissance (87) qu'ils (54) qu'on (57) religieuse (59) religieux (63) religion (108) seigneur (67) sol (96) temps (98) terre (80) vertu (76) vie (168)

created at TagCrowd.com

- A Typical Mission in China, by William Soothill (eventually he was to become Professor of Chinese at Oxford), 1907

able (59) book (71) case (72) china (314) chinese (226) christian (126) church (170) city (93) country (66) days (62) death (59) earth (71) form (65) god (173) head (60) heaven (125) home (101) house (70) hundred (60) indeed (70) learn (65) life (140) man (381) means (102) mission (199) missionary (98) native (129) others (78) people (167) poor (65) power (72) preacher (73) present (81) read (61) religions (105) school (70) sermon (60) service (62) son (81) spirit (63) temple (65) things (69) thousand (71) wife (60) women (60) work (152) world (84) worship (75) years (258) young (62)

created at TagCrowd.com

This cloud being taken from an English text is also supposed to gather words by proximity (extra-function).
Here is the list of non-wanted words so that I could have a "cleaner" cloud, with more "meaningful" words: ago, already, away, become, became, found, mr, name, number, soon, whose, brought .

As we can see this process would probably not satisfy a demanding cliometrician, or linguist, or historian (on what basis do I eliminate the verb "become", belonging to the lexical realm of change?). And other softwares exist that would. But this web app is free, reliable, adaptable, easy to use and produces pretty clouds of visual significance. When will a Chinese version be available?


PS: The tags for this post were TagCrowd generated....