Do you remember in high-school where the more friends you had, the easier it was to make more friends?  Link building on the internet applies the same phenomenon.  If you are able to a stronger site then yours build a link to you then more and more sites would be more willing to build links to you.  The more links you build, the more popular your site becomes with the search engines.  This favorably affects your page rank and the likelihood of appearing higher and higher on organic searches. 

It does not just take having random links built around you.  Just like the high school example, you try to maintain friends that share similar interests to you.  You do get judged by the friends you keep.  Similarly, with advancements in search engine algorithms, these search engines check the content of sites linking to you relative to your content.  If they see a large number of sites that link to you directly related to what you do, they weigh you higher in page ranking.  Conversely, if you have a large number of random links created to your site, the search engines don't know where you should fit and you may often appear in non-related search results--effectively reducing your conversion rates.