Based on responses from more than 2,000 US smartphone, tablet and desktop users, the 2014 Digital Wallet Usage Study found security to be their leading concern (46%).
Finding it easier to pay by cash or card is the second main reason for not using digital wallets (37%), followed by not considering the option (32%), said Thrive Analytics, which released the survey.
Jason Peaslee, managing partner at the Ohio-based marketing consultancy, said this presented a "vast opportunity" for digital wallet providers and retailers to educate consumers about their benefits.
"The key to increasing consumer and overall market adoption is understanding and operationalising critical demographic usage profiles and purchase patterns," he said.
For example, the survey found half of all consumers carry less than $20 on a regular basis and three-quarters of those who carry no cash at all are aged under 40.
PayPal is by far the most-used digital wallet, at 79%, followed by Google Wallet (40%), Groupon (26%) and Apple Passbook (17%).
Women tend to use apps from stores like Macy's to obtain coupons and discounts while men use them mostly for browsing and service related activities.
The survey coincides with news that internet retailer Amazon will be launching a new app, Amazon Wallet, which allows users to store loyalty cards or gift cards, the Wall Street Journal reported.
The app will be preinstalled on its Fire smartphone, which is set to be released on Friday.
Data sourced from Thrive Analytics, Wall Street Journal; additional content by Warc