WHY THEY EXIST
In order to protect normal people, the SEC has created all sorts of rules and regulations for how companies that invest money on behalf of other people should operate. While this makes the investments safer and less volatile, it prevents the firm making investments from chasing riskier but possibly more profitable investments.
WHAT ARE THEY
Hedge funds are companies that make these investments. They are not allowed to have more than 100 investors, and they are not allowed to take on any investors with less than $5 million in wealth. Many of their clients are either very wealthy people or institutional investors, such as pension funds, banks, insurance companies, college endowments, or sovereign wealth funds. It's also very difficult to get into a good hedge fund, as not only are they limited by the number of investors they can take on, but they are also rated on the rate of return they earn on assets, and the more money you have to invest, the more opportunities you need to find. If someone offers to let you invest in their hedge fund, remember the Groucho Marx joke about not wanting to join a country club that would have him as a member.
GOALS
So in short, hedge funds gather large piles of money from very wealthy institutions and invest it on their behalf. The goal is to do two things. The first is to demonstrate alpha. To explain this concept, think of it this way. If you hear that the stock market went up by 10%, and you look at all the stocks on the market, odds are almost all of them went up. A rising tide lifts all boats. This is referred to as beta. Most assets, or portfolios of assets, will move in the direction of general market. Alpha is finding assets that go up even more than the general market. Anybody can just buy into a fund through say Vanguard, and own the broad market index, and get the above mentioned 10%. People invest in hedge funds to earn an amount above and beyond that 10%. Alpha is supposed to be the ability to find assets that will do this.
The other goal of hedge funds is to earn absolute returns. What this means is that they make money every year, regardless of what the stock market does. A few funds have done this, but 2008 demonstrated that most funds were bluffing in saying they were able to do that, and many of them went out of business.
HOW DO THEY SET IT UP
So based on the above, the hedge fund has a large pile of money, millions or sometimes even billions of dollars, invested with it by various clients. A hedge fund is normally staffed by veterans of the financial services industry, typically people who have worked for a long time at investment banks and have contacts within the industry. After accumulating their investment capital, their frequently turn to investment banks, such as Goldman Sachs, Merrill Lynch, or Morgan Stanley, or commercial banks with investment banking arms, such as Citigroup or Bank of America, to borrow large sums of money to leverage this return. What this means if that if you invest $100 and make 10%, you make $10. But if you borrow $900, and invest the full $1,000 and make 10%, you make $100, then pay back the bank $900 (plus interest), and you made $100 (less interest) for your investors off the $100 they invested with you. (The downside of this is that if you lose 10% on your $1,000 investment, you're down to $900, and your investors are wiped out. This is called blowing up in the jargon, and it happens more than you would think, so leverage can bevery dangerous). The banks like to lend to hedge funds though, because they like the interest income, and they like to hear what leads the hedge funds are following (financial services is very incestuous, and rumors drive trades). The hedge funds like to borrow from the banks because, notwithstanding the need for leverage, the investment banks have contacts within the publicly traded companies. Say you're doing a bond offering through Goldman Sachs. Maybe in the process of that, you meet their hedge fund clients, and talk up your business to them to drive demand for your stock, while they hear from the CFO and CEO personally, and make better decisions than the average investor pouring over financial statements.
WHAT DO THEY INVEST IN
OK, so now we have a bunch of guys who used to work for investment banks, with investor money, contacts at investment banks, and large piles of borrowed money. What do they invest in? The short answer is whatever they want. There are a number of recognized investing styles though.
1) Long short - These are the most simple funds. They buy some stocks, and short other stocks. The desire is to assemble a portfolio that will earn both above average and absolute returns. The concept of going long and short, done to hedge risks, is how the industry got its name back in the 1950s.
2) Global Macro - These funds invest in assets that will give them exposure to broad economic forces, which they hope to predict. Do you think the euro is going to break up? Short it, and euro-denominated assets. Do you think we've hit peak oil? Buy oil companies, and oil futures. George Soros became famous for making billions for predicting that England would be forced to devalue its currency in the 1980s.
3) Directional - These tend to be referred to as "black box funds", meaning that no one knows what their criteria are, as it is a trade secret. Often they write computer programs designed to go through millions of pages of data, and analyze all kinds of data to find mispriced assets and go long or short on them. High frequency traders do this with stocks by setting up a computer near a stock exchange that can buy stocks and sell them again within nanoseconds. I also read about a fund that combs through publicly available filings through the Food and Drug Administration, looking for any changes regarding new drugs, to use as information on pharmaceutical stocks.
4) Event Driven Funds - As the name suggests, the buy or sell short assets whose price will be affected by future events. Merger arbitrage is a common form of this. When one company acquires another, usually the buyer's stock goes down, as they are probably overpaying, while the acquired company's stock jumps, as someone is about to overpay for it. Short the predator and buy the prey. In the movie Wall Street, Charlie Sheen uses contacts with a law firm to find clients of the firm who are planning mergers to conduct this sort of strategy, and is arrested when the feds flip one of his accomplices. The most famous example of this strategy is John Paulson, who made billions off the housing bubble buying credit default swaps on collateralized debt obligations (CDOs). The Big Short by Michael Lewis also analyzes funds that took this approach. (I believe Paulson is now following a macro strategy, and has been buying up gold mines, evidence that funds can change strategies).
5) Fund of funds - This is a hedge fund that invests its money in other hedge funds. By diversifying among various funds, the hope is that returns will be less volatile and safer. If one fund blows up and loses all its money, that might be 5% of your investment rather than 100%.
There is a pretty long list of other recognized strategies, many too complicated to worry about here. (For example, the Galleon hedge fund, which was in the news recently, was using contacts in consulting firms and other places to obtain and trade on inside information. The founder is now facing 15 to 19 years in prison. It was an effective strategy, but one that obviously had severe consequences.) But the point of this is to indicate that they can invest in whatever they want, and will be judged solely on how much money it makes.
HOW DO THEY GET PAID
You've probably already heard about how many hedge funds make billions in profits. Hedge funds get paid in two ways. The first is the management fee. If you invest money with a hedge fund (or anyone for that matter), the first thing they do is take a percentage of your money from you and keep it for themselves. Hedge funds typically charge 2% of assets under management each year. They use this money to pay employees, rent office space, and cover overhead until their real payday arrives at the end of the year.
The other means of payment is the performance fee. At the end of the year, the fund tallies how much money it made for investors, and takes a chunk of it for themselves. 20% is typical, but amounts have varied. So if the fund makes $5 billion for its investors, it is a $1 billion payday for the people who run the fund. If you have a lot of money under management, and earn high rates of return, you an become fantastically wealthy very quickly.
PROBLEMS WITH HEDGE FUNDS
One thing you have to hand to hedge funds is that in 2008 and 2009, when investment and commercial banks were all begging for bailouts from the federal government because they were all "to big to fail", thousands of hedge funds died fast and anonymous deaths. Many of them lack the wealth, political connections, and systematic importance to influence the government. If people want to gamble with their money in financial markets, that isn't problematic as long as they are willing to suffer the losses they incur. (It is worth noting that money of them benefited indirectly from the various bank bailouts, and especially the bailout of AIG. Much of the bailout of AIG was to pay off credit default swaps (CDS) on mortgage backed securities, and the proceeds of those CDSs went to certain hedge funds.)
There has been some concern that some of the larger hedge funds could be systematically important, and could cause problems in the future if they place unmitigated bets in financial markets. The Dodd Frank Act made some effort to control them by having them register with the SEC and provide information on their bets to regulators, but as usual the regulation is watered down and many will comply with the letter but not the spirit of the law.
Also, in reviewing the results of most hedge funds, the investors don't seem to be getting a very good deal. First, it's incredibly expensive to pay someone 2% of your money up front and give them 20% of all returns that they earn on the remaining 98% of your money. Many investors would have been better off investing in more conventional vehicles, but the allure of investing in a hedge fund combined with the possibility of outsized returns lures in a lot of people.
The other risk of hedge funds is that they are frauds. Bernie Madoff was running a long short hedge fund, but it was actually a Ponzi scheme. A number of fund of funds were supposed to be diversifying their investor's assets, but were actually just giving it all to Madoff. (It's pretty insulting to pay someone to invest money for you, and give them 2% plus 20%, only to have them dump it all in a Ponzi scheme after performing no due diligence.) Another form of fraud might just be taking on asymmetrical risk. For example if, like AIG, you sell tons of insurance on the housing market, you can pocket a lot of money in the hopes that you never have to pay up on this insurance, and if you do, you just make vague statements about "once in a lifetime" financial calamities.
TL;DR: They are investment vehicles for rich people and institutions that invest in whatever they think will make money. They're supposed to make their investors a lot of money, but they definitely make themselves a lot of money.