The Zipf Mystery

15.09.2015
18 976 817 Näkymät

The of and to. A in is I. That it, for you, was with on. As have ... but be they.
RELATED LINKS AND SOURCES BELOW!
twitter.com/tweetsauce
instagram.com/electricpants
WordCount.org www.wordcount.org/
How many days have you been alive? www.beatcanvas.com/daysalive.asp
random letter generator: www.dave-reed.com/Nifty/randSeq.html
Dictionary of Obscure Sorrows: fiworld.info
Word frequency resources:
[lemmatized] en.wikipedia.org/wiki/Most_common_words_in_English
www.uow.edu.au/~dlee/corpora.htm
www.wordfrequency.info
www.anc.org/data/anc-second-release/frequency-data/
www.titania.bham.ac.uk/docs/
www.kilgarriff.co.uk/bnc-readme.html#raw
en.wiktionary.org/wiki/Wiktionary:Frequency_lists
ucrel.lancs.ac.uk/bncfreq/
[PDF] www.wordfrequency.info/files/entries.pdf
[combined Wikipedia and Gutenberg] www.monlp.com/2012/04/16/calculating-word-and-n-gram-statistics-from-a-wikipedia-corpora/
corpus.byu.edu/coca/files/100k_samples.txt
corpus.byu.edu/
corpus.leeds.ac.uk/list.html
books.google.co.uk/books?id=ja1_AAAAQBAJ&dq=word+frequency+coca&lr=
www.ling.helsinki.fi/kit/2009s/clt231/NLTK/book/ch01-LanguageProcessingAndPython.html
Great Zipf's law papers:
colala.bcs.rochester.edu/papers/piantadosi2014zipfs.pdf
www.ling.upenn.edu/~ycharles/sign708.pdf
arxiv.org/pdf/cond-mat/0412004.pdf
www-personal.umich.edu/~mejn/courses/2006/cmplxsys899/powerlaws.pdf
Zipf’s law articles and discussions:
www.theatlantic.com/magazine/archive/2002/04/seeing-around-corners/302471/
io9.com/the-mysterious-law-that-governs-the-size-of-your-city-1479244159?.Ej9760cOTJCPS_Bq4mjoww.0
plus.maths.org/content/os/latestnews/may-aug08/food/index
judson.blogs.nytimes.com/2009/05/19/math-and-the-city/?em
plus.maths.org/content/mystery-zipf?src=aop
www.datasciencecentral.com/profiles/blogs/why-zipf-s-law-explains-so-many-big-data-and-physics-phenomenons
en.wikipedia.org/wiki/Zipf%27s_law
books.google.co.uk/books?id=f8GrzlnMSm8C&pg=PA62&redir_esc=y#v=onepage&q&f=false
arxiv.org/pdf/0802.4393v1.pdf
www.pnas.org/content/108/9/3526.full
lewisdartnell.com/language_page.htm
wugology.com/zipfs-law/
other Zipf’s law PDFs
ftp.iza.org/dp3928.pdf
arxiv.org/pdf/1402.2965.pdf
arxiv.org/pdf/1104.3199.pdf
www.lel.ed.ac.uk/~jim/zipfjrh.pdf
www.ncbi.nlm.nih.gov/pmc/articles/PMC2834740/#pone.0009411-Mandelbrot1
polymer.bu.edu/hes/articles/pgs02a.pdf
in untranslated language: arxiv.org/pdf/0808.2904.pdf
pages.stern.nyu.edu/~xgabaix/papers/zipf.pdf
www.hpl.hp.com/research/idl/papers/ranking/ranking.html
statweb.stanford.edu/~owen/courses/306a/ZipfAndGutenberg.pdf
arxiv.org/pdf/1310.0448v3.pdf
www.kornai.com/Papers/glotto5.pdf
Zipf’s law slides:
www.slideshare.net/guest9fc47a/nlp-new-words
Pareto Principle and related ‘laws’:
www.squawkpoint.com/2013/03/pareto-principle/
billyshall.com/blog/post/paretos-principle
en.wikipedia.org/wiki/Pareto_principle
Random typing and Zipf:
www.longtail.com/the_long_tail/2006/09/is_zipfs_law_ju.html
health 80/20: archive.ahrq.gov/research/findings/factsheets/costs/expriach/expriach1.html
Principle of least effort:
en.wikipedia.org/wiki/Principle_of_least_effort
en.wikipedia.org/wiki/Satisficing
www.pnas.org/content/100/3/788.full.pdf [PDF]
csiss.org/classics/content/99
self organized criticality:
journal.frontiersin.org/article/10.3389/fnsys.2014.00166/full
Hapax Legomenon:
campus.albion.edu/english/2011/02/15/hapax-legomenon/
www.dailywritingtips.com/is-that-a-hapax-legomenon/
en.wikipedia.org/wiki/Hapax_legomenon
[PDF] www.aclweb.org/anthology/J10-4003
www.wired.com/2012/01/hapax-legomena-and-zipfs-law/
oed.hertford.ox.ac.uk/main/content/view/402/450/index.html#_ftn1
oed.hertford.ox.ac.uk/main/content/view/36/166/index.html
Learning curve: en.wikipedia.org/wiki/Learning_curve
Forgetting curve:
www.trainingindustry.com/wiki/entries/forgetting-curve.aspx
en.wikipedia.org/wiki/Forgetting_curve
Experience curve effects: en.wikipedia.org/wiki/Experience_curve_effects
Forgetting
and zipf's law: act-r.psy.cmu.edu/wordpress/wp-content/uploads/2012/12/37JRA_LS_PS_1991.pdf
public.psych.iastate.edu/shacarp/Wixted_Carpenter_2007.pdf
marshalljonesjr.com/youll-remember-less-than-001-of-your-life/
en.wikipedia.org/wiki/Forgetting
www.reddit.com/r/Showerthoughts/comments/3gu9qk/it_only_takes_three_generations_for_you_to_be/
music from:
fiworld.info
www.audionetwork.com

Kommentteja
  • Most things go Top to bottom, you don't start a cake with a "hint of cinnamon " or " a cherry" even someone doing there accounts would start with big number first.

    Robcop WesttonRobcop Westton40 minuuttia sitten
  • would have been funny if he didnt say the for the entire video

    Pigbelly19 ‘Pigbelly19 ‘2 tuntia sitten
  • mfw i suddenly realise the background music at 12:45 is actually bach double but jaunty piano mode

    Jacob CostenJacob Costen3 tuntia sitten
  • Yeah its simple.

    ِِ3 tuntia sitten
  • Where did I put my paper clips?

    Toms MarkovsToms Markovs4 tuntia sitten
  • Answer next tk questions

    ِِ4 tuntia sitten
  • A similar law is Benford's Law.

    Vineet SharmaVineet Sharma5 tuntia sitten
  • me and the bois going on to wikapedia and making scauce the 69 most used word

    Henry AndersonHenry Anderson5 tuntia sitten
  • Reading up on superspreading events and epidemiology, the 80/20 rule was also observed -- 80% of disease transmission is caused by 20% of infected cases

    TheMockingJayTheMockingJay7 tuntia sitten
  • there is no way "school" is uses more than "no"

    Cameron GoodallCameron Goodall18 tuntia sitten
  • The Bible was written by man bc it follows ziffs law

    blocboy jbblocboy jb20 tuntia sitten
  • we bouta get quizzacious

    Matt GarreltsMatt Garrelts21 tunti sitten
  • This is Benford's law

    Andrej AjnikAndrej AjnikPäivä sitten
  • The absent reason putatively guide because transaction regretfully milk besides a overrated territory. aboard, tranquil skirt

    Andrew ZeitlerAndrew ZeitlerPäivä sitten
  • Sounds very similar to Benford's law also. There seems to be a more primary factor causing the occurrence of this throughout various systems. Living in a simulation becomes more plausible the more you realize the patterns.

    Dennis GarageDennis GaragePäivä sitten
  • I think this all comes down to usefulness and meaning. The could be used to pinpoint any other word while sauce is so specific that it isn’t used as much because the explanation is direct and intentional. The is what exactly? but sauce is sauce.

    Lu BuLu BuPäivä sitten
  • 80% of my stories are filled with um and 20% of my stories are actually told

    Jacob PackmanJacob PackmanPäivä sitten
  • Dude, after 2 video spirals on your channel I'm not ashamed to admit that although chunks of the content my idiot brain has no way to comprehend it. But the way you present and explain the info, what I end up being able to fathom, blows my mind! Love your content.

    John HeggertyJohn HeggertyPäivä sitten
  • Bruh

    Shard FloofFoxShard FloofFoxPäivä sitten
  • We live in a simulation.

    Drooby Dooby DooDrooby Dooby DooPäivä sitten
  • Who else tried to count the amount of times he said "the"

    Tuckster04Tuckster04Päivä sitten
  • This 80/20 split forecasted Tfues contract

    Edward TrevinoEdward TrevinoPäivä sitten
  • It indicates that there is one creator of the universe.. The One and Only God

    Aljulanda AlmaghadariAljulanda AlmaghadariPäivä sitten
  • All I see is FRACTALS.

    cant runcant runPäivä sitten
  • 3:03 wow

    Kjonesey ‘Kjonesey ‘2 päivää sitten
  • 80 percent of my test is about 20 percent of what we had to study

    Elise VisserElise Visser2 päivää sitten
  • *s a u c e*

    Doby AdobyDoby Adoby2 päivää sitten
  • Meanwhile: German *der die das die den die das die dem der dem den des der des der*

    a aa a2 päivää sitten
  • since this cideo was published, Quizzaciously has gone from 1 result, to 17,500 results. Proving your point that the more something is used, the more it be used again.

    Ryan CooperRyan Cooper2 päivää sitten
  • 2:08 normal people: wow people really like sauce. me being a weeb: ( ͡° ͜ʖ ͡°)

    Nooby-kunNooby-kun2 päivää sitten
  • Zipf's Law is just the algorithm that God put into a lot of shit after a long day of developing the " everything"

    Cedrick NarsisoCedrick Narsiso2 päivää sitten
  • Vsauce always makes me feel philosophical

    toastboiitoastboii2 päivää sitten
  • my grandma did a funny dance on the floor and i dont know why vsauce help

    EE2 päivää sitten
  • in the beginning was the word, and the word was with God and the word was God

    papinbalapapinbala2 päivää sitten
  • Is this proof that we’re in a simulation?

    James TaylorJames Taylor2 päivää sitten
  • This law explains why you can never solve the problem of inequality in every single measurement of socio-economic status.

    pizzasmithdudepizzasmithdude2 päivää sitten
  • I think that Zipf was correct. Speaking more than ten languages myself, I've come to the theory that language is just a way to transfer thoughts. In doing so you will need a (rare) specific word and several (common) words to describe how people should interpret the specific word.

    Mark PrinsMark Prins3 päivää sitten
  • "[...] in *the* english language. About one in every 16 words you encounter on a daily basis is *the* ." son of a...

    Lady YmirLady Ymir3 päivää sitten
  • Hapax Legomenon derives from the Greek Άπαξ λεγόμενον which literally means “ said once”.

    Deconstruct//RecreateDeconstruct//Recreate3 päivää sitten
  • so then what happens when 99% of the wealth is with 1% of the people. such a huge imbalance im sure Cthulu will come and have to balance that out. or maybe thats why the sun is freaking the fuck out and is about to spit a micro nova at us.

    Brave CatBrave Cat3 päivää sitten
  • michael saying "80 percent and..." so much fucked me over ngl

    KUWALDAKUWALDA3 päivää sitten
  • It is funny how the Pareto principle is relevant today. Here in Quebec, Canada, they said that 20% of the people at risk of contracting the COVID-19 virus (mostly people over 65 years old) represent 80% of the people hospitalized in our hospitals! I didn't know about that principle then!

    AkylliaAkyllia3 päivää sitten
  • too bad i can't check out wordcount since flash player is dead

    QwertyQwerty3 päivää sitten
  • Kinda puts the kibosh on the whole conservative assertion that the wealthy have what they have because they "deserve it". Seems more likely that they're just lucky beneficiaries of some cosmological constant that had little to nothing to do with their effort. This of course begs the question: if you're no more entitled to wealth or power than the next person, *why do we constantly accept systemic wealth and power inequality?*

    Mr. EMr. E3 päivää sitten
  • "Zipf-y"

    Will It Snap?Will It Snap?3 päivää sitten
  • Vsauce Say "I want" and then everything after it without (The)

    Junovah HunziilJunovah Hunziil3 päivää sitten
  • What a video!!!

    Complex GodComplex God3 päivää sitten
  • Video must be old. The most common word now is literally :p

    Scott KeselowskiScott Keselowski3 päivää sitten
  • everyone use the second most used word as much as possible

    fewbronze gamesfewbronze games3 päivää sitten
  • This can't be right ....

    Itchy Eyes McRealizeItchy Eyes McRealize3 päivää sitten
  • I know use quizatiously once at least a day I no speel

    Joe BlackmoreJoe Blackmore3 päivää sitten
  • Well order is the last thing that you'd think chaos would do, so thats exactly what it did

    The Failed_safeThe Failed_safe4 päivää sitten
  • Have you forgotten this video now that 5 years have passed?

    Hugo Le RoyHugo Le Roy4 päivää sitten
  • He got us in the first half, not gonna lie.

    SoftAlexandra3 PlaysSoftAlexandra3 Plays4 päivää sitten
  • 0:26 This is a really very good Poem.

    Helpful SysopsHelpful Sysops4 päivää sitten
  • "The" dont exist in my language. You change the word instead that wold be after "the".

    RobinRobin4 päivää sitten
    • 1. Interesting, which language is that? 2. Languages don't necessarily agree on the order. Inuits probably have many words that are frequently used like snow/white/cold, while the Bedouin probably have very few words like that. On the other hand, Bedouins probably have and use many words related to yellow/sand/hot, while the Inuits probably use these rarely if at all.

      Sander BouwhuisSander Bouwhuis3 päivää sitten
  • This is like the Fibbonaci Sequence.

    Ethereal Sky DaddyEthereal Sky Daddy4 päivää sitten
  • What if we tend to use more words than others because some might feel repetitive? If someone puts "the" 30 times in an essay it will certainly look better than if they put "furthermore" also 30 times in it.

    Sebastian SolisSebastian Solis4 päivää sitten
  • Your science is flawed as only a few letters on their own are actually words. F or Q are not words for example, they are letters.

    tdchewytdchewy4 päivää sitten
  • After the intro his percentage when the up to 7%

    [GD] Goose[GD] Goose4 päivää sitten
  • Im scared of this \ line now

    Cattu DogguCattu Doggu4 päivää sitten
  • 14:08 imagine this in a bernie sanders voice LOL

    Mag MagnetMag Magnet4 päivää sitten
  • Okay but Vilfredo Pareto has to be the coolest name ever

    JayleighJayleigh4 päivää sitten
  • every time micheal says the word "the" I go: "Micheal stop! STOP! You're killing me!"

    Double A-AADouble A-AA4 päivää sitten
  • me: how is this possible? Vsause: yes

    Eric YoungEric Young4 päivää sitten
  • V-sauce. Really? "Its like if you take a snowball and roll it down a snowy hill." Yes my genius friend, the snowball effect.

    Benjamin CoffmanBenjamin Coffman4 päivää sitten
    • Still love you Michael :p

      Benjamin CoffmanBenjamin Coffman4 päivää sitten
  • Haha the

    Liam BinsfeldLiam Binsfeld5 päivää sitten
  • So, quizzaciously is a fun word, but can it beat fabaceous (Adv. of the nature of the bean)?

    Eye Master 3.14Eye Master 3.145 päivää sitten
  • Quizzaciously

    InsteadImAwakeInsteadImAwake5 päivää sitten
  • That intro is what makes time continue

    qwzerty_qwzerty_5 päivää sitten
  • The better quart enzymatically whistle because flugelhorn likely shade under a onerous care. boring, mushy link

    Bernice GardnerBernice Gardner5 päivää sitten
  • 11:27 top ten anime plot twists

    Ultimate FlanUltimate Flan5 päivää sitten
  • This guy is extraordinary

    afeelfortheflow with Manicafeelfortheflow with Manic5 päivää sitten
  • AAAAAAAA! Vsauce is making me interested in math! NOOOOO What do I do lol

    Magic BeansMagic Beans6 päivää sitten
  • so if i study onlky 20% of most used words & grammar i can learn 80% of a language 🤑

    CieloCielo6 päivää sitten
  • This is like almost actual proof we are in a simulation

    A1 HelixA1 Helix6 päivää sitten
  • Since there are an infinite causes and effect pairs, I believe every arbitrary combination like 30/70 or 50/50 will also have infinite occurences (there may not be anything special about 20/80). If someone shows that the popularity or frequency of a particular combination, say 20/80, is more compared to the rest of the possibilities, then it would be something weird. Listing a 100 examples which follows 20/80 principle from an infinite more total examples does not lead me to believe that this combination is special. Nice video though, as always 😉.

    Vivek SaraswatVivek Saraswat6 päivää sitten
  • We broke the law!!! Only 9 percent of plastic is recycled 91 percent is in landfills or is litter.

    Thomas BeasleyThomas Beasley6 päivää sitten
  • 80% of my life is boring but 20% is more boring.

    {DavidDoesn'tKnow}{DavidDoesn'tKnow}6 päivää sitten
  • We have the Illuminati all figured out now 😏. They thought they could pull the Zipf over our eyes. Not so much anymore.

    THE JACK HQTHE JACK HQ6 päivää sitten
  • 4:23 George Zipf é a cara do jornalista Augusto Nunes.

    Hugo PetrarolhaHugo Petrarolha6 päivää sitten
  • Vilfredo Pareto

    Eli PhelpsEli Phelps6 päivää sitten
  • i didnt understand any of that

    Emlar TubbsEmlar Tubbs7 päivää sitten
  • Of all the stuff I have 20% is useful, 80% is junk..

    RainManRainMan7 päivää sitten
  • the word "um" is used more than "the

    DynaBloxDynaBlox7 päivää sitten
  • 80% of my code sucks and 20% of my code sucks whats the problem then

    Harvey YFBYHarvey YFBY7 päivää sitten
  • It made me sure that I am living in a stimulation: (

    yash Payalyash Payal8 päivää sitten
  • 80% of micheals hair is on 20% of his head

    Cesar ArizpeCesar Arizpe8 päivää sitten
  • maybe because we all run in one ai program

    GodGod8 päivää sitten
  • How can you not translate the unknown language, all ya gotta do is figure out what's used the most then that must be the word "the" and so on.

    Carla MaciasCarla Macias8 päivää sitten
  • All this tells me is that life is actually just a lot of annoying math problems I don't want to do

    jisdaboss1jisdaboss18 päivää sitten
  • Why do I only watch vsauce at 5 in the morning

    That GuyThat Guy8 päivää sitten
  • Zipfs law seems like an Easter egg the creators of the world put in when they were designing us, in the simulation, we're just part of a simulation created by other beings and they made things like zipfs law intentionally for us to eventually find. Imagine the things we don't yet know, or even better imagine the things we think we know but are just based off of speculation

    Jonathan SheddJonathan Shedd8 päivää sitten
    • Speculation? Oh, you mean like speculation as in there must be a creator

      BFB-DanceySteveBFB-DanceySteve7 päivää sitten
  • He is so quizacious

    Alex WallAlex Wall8 päivää sitten
  • Someone should do this for the 3 holly books. Just for fun.

    Tarik TezcanTarik Tezcan8 päivää sitten
  • Cycloid ... 🙌🏽

    Dead ManDead Man9 päivää sitten
  • @Vsauce

    Asif MolviAsif Molvi9 päivää sitten
  • @vsauce please make a video of benfords law

    Asif MolviAsif Molvi9 päivää sitten
  • I got through about half of this video.

    xxxgammaxxxxxxgammaxxx9 päivää sitten
  • That was until this new generation. Now it seems that every other word is "like".

    billy watkinsbilly watkins9 päivää sitten
FIworld