Alibaba Group Holding Ltd. has invested $215 million in Silicon Valley startup TangoMe Inc., giving the Chinese e-commerce giant access to a formidable competitor in the mobile-messaging race as it prepares to hold an initial public offering in the U.S.

The minority investment is part of a $280 million financing round that values the four-year-old company at more than $1 billion, according to a person familiar with the deal. Alibaba was the only new investor in the round, which was joined by some of Tango's previous venture-capital backers.

Tango is one of the more-popular apps used to make free video calls, like Microsoft Corp.'s Skype and Apple Inc.'s FaceTime, though it has evolved into a broader messaging service with texting, photo sharing and games.

Tango's user base is much smaller than messaging apps like WhatsApp, recently acquired by Facebook Inc., and WeChat, owned by Alibaba's rival Tencent HoldingsLtd. But its membership has doubled over the past year to more than 200 million registered users, with about 70 million returning to the app each month. WhatsApp has more than 450 million monthly active users, and WeChat had 355 million at the end of last year.

Yahoo co-founder Jerry Yang made the introduction between Tango and Alibaba. Mr. Yang, one of Tango's early investors, also knew Alibaba because he led Yahoo's prescient investment in the e-commerce company.

In the past year, Alibaba has been investing large sums of capital in key areas of mobile technologies such as social media, mapping and e-commerce to bring more users to its two online marketplaces, Taobao and Tmall. The majority of transactions taking place on those two platforms still come from PCs.

Last year, Alibaba bought an 18% stake in Sina Corp.'s Twitter-like microblog Weibo, which on Friday filed for an IPO in the U.S. Products listed on Taobao are now available for purchase on Weibo.

Alibaba also bought a 28% stake in Chinese mobile-mapping firm AutoNavi Holdings Ltd., and has begun working with the company to share user data. Eventually, it could track user movements in order to be able to link them to nearby services and pipe in e-commerce ads based on their location.

Its own messaging app, Laiwang, is aiming to compete with WeChat, but it has far fewer users. In November, a month after launching, Alibaba said Laiwang had more than 10 million users.

Last summer, Alibaba began blocking merchants on Taobao and Tmall from using WeChat to send promotional messages to shoppers. Tencent, meanwhile, added mobile payments last year and has been offering more services and shopping options on WeChat.

There are no immediate plans to connect the Tango app with the company's various e-commerce services, though that could come over time, said a person familiar with the plans. Tango will operate separately from Laiwang.

Tango also expands Alibaba's portfolio of fast-growing U.S. startups as the Chinese company tries to gain a foothold here. It led investments last year in app search-engine Quixey Inc. and two-day shipping provider ShopRunner Inc.

On Sunday, Alibaba said it started the process of listing on a U.S. stock exchange, a first step toward what could be the largest Internet IPO in history.

Alibaba's cash will help Tango, based in Mountain View, Calif., to double its workforce to more than 300 this year and open up its service to more outside developers, Chief Executive Uri Raz said.

Mr. Raz and his business partner Eric Setton co-founded Tango in 2009 as a way to connect with family members overseas. The two had previously built video-streaming startup Dyyno and wanted to apply their expertise to video calling.

Mr. Raz said he began adding other services, such as games and photo sharing, after seeing the success of Asian texting apps such as WeChat, Kakao and Line. "We were looking at what happened in Japan and China," Mr. Raz said. "It was the first time since 1999 that I saw the speed of innovation happening faster over there than here."

Tango joined with outside developers, such as game maker Gameloft Inc. and music-streaming service Spotify AB, to offer users a range of virtual activities inside the app. For example, Tango users can share 30-second clips of their favorite songs from Spotify or compete with their friends for the high score in the Gameloft title "Candy Block Breaker." The company shares the revenue from purchases made inside the apps with the developers.

By the end of this year, Tango plans to open its software code to allow any outside developer to build games and other services for its users, Mr. Raz said.

The startup is also bolstering its revenue through mobile advertising. Last December, it began working with Twitter Inc.'s MoPub unit to sell ads within users' news feeds and message inboxes, promoting apps made by Zynga Inc. and others.

The app is most popular in the U.S., Western Europe and the Middle East, Mr. Setton said, with about three-quarters of the users spread evenly across those regions.

While the company has an office in Beijing and has seen rising usage in Thailand and Hong Kong, Asia isn't currently a main area of focus for the company, Mr. Raz said.

Mr. Setton says the alliance with Alibaba will give Tango market insights that will help it beat rivals to new trends.

Rather than take on WeChat in China, Mr. Raz said he's more focused on perfecting the model for U.S. users.

"We want to build the WeChat of the Western world," Mr. Raz said. "We have all the WeChat features but each one of them is Westernized."

—Paul Mozur contributed to this article.

Write to Douglas MacMillan atdouglas.macmillan@wsj.com