An international survey of 600 global data leaders conducted by Informatica shows that data leaders in the Asia-Pacific region are seriously promoting the promotion of artificial intelligence. India leads the region by a wide margin, with 75% of respondents already adopting generative AI.
However, respondents in Asia Pacific face barriers to AI data management. These include data fragmentation from the explosive growth in the number of data sources, data quality that can be used for artificial intelligence, and embedded data governance sufficient to meet the challenges of artificial intelligence.
Richard Scott, senior vice president of Informatica Asia Pacific and Japan, said data literacy is very important to support organizational data management. Scott recommends adopting a cloud data architecture from the beginning and focusing on people, process and technology.
Artificial intelligence is driving a parallel focus on data management
APAC data leaders say the ability to provide reliable and consistent data for generative AI (40%) is a top priority for data strategy in 2024, along with improving data governance and processes (40%). This shows that artificial intelligence is driving a common focus on data management.
See: Top 10 benefits of improving the quality of your organization’s data.
The close connection between artificial intelligence and data is also reflected in investment willingness. Three-quarters (78%) of Asia Pacific data leaders predict their data investments will increase in 2024. No respondent does not plan to invest in data management capabilities in some form.
Regional investment in critical data capabilities continues to grow
Many data management capabilities are receiving investment based on data strategy priorities. Data privacy and protection was rated No. 1 (45%), reflecting the need to maintain data privacy and security in a rapidly changing cybersecurity environment.
This is followed by data quality and observability (42%) and data integration and engineering (40%).
“We’re seeing continued improvements in the focus areas of data quality and data governance,” Scott said. “So AI is really going to drive a new wave of data cleansing.”
See: How IBM’s Matthew Candy sees Australia’s pursuit of generative AI scale in 2024.
Artificial Intelligence poses many data management challenges
According to Informatica’s global survey results of data leaders in organizations with over $500 million in revenue, nearly all (99%) data leaders have encountered obstacles in their AI journey, including data leaders in Asia Pacific By.
Data fragmentation and data growth
Asia Pacific data leaders expect data fragmentation and complexity to worsen in 2024. Informatica found that 56% of data leaders are struggling to balance more than 1,000 data sources. Additionally, 78% of Asia Pacific data leaders expect the number of data sources to increase this year.
“Last year alone, Informatica processed approximately 86 trillion cloud transactions per month, a 60% increase over the same period last year,” Scott explained. “So even though organizations are trying to get their data warehouses organized, data is still exploding; we’re seeing this really explosive growth,” he said.
Data quality and artificial intelligence model bias
42% of global respondents cited data quality as the top challenge in generating artificial intelligence. Potential bias is of particular concern in the Asia-Pacific region due to the large size of language models; 53% of Australian respondents said avoiding bias was their top concern (Figure A).
“In the age of analytics, if you don’t have a good data foundation, you’re going to make bad decisions faster,” Scott said. “Similarly, if your data management environment is not good, you will get answers from generative AI, but it may lead you in the wrong direction.”
Data literacy beyond data assets
Data leaders surveyed said organizational data literacy is hindering the advancement of artificial intelligence. For example, 98% of global data leaders say they encounter non-technical organizational barriers to better data management, such as a lack of leadership support.
39% of global data leaders see improving their data-driven culture and data literacy as a top priority in 2024. Improving data literacy is the second most important indicator of data strategy effectiveness (42%), behind data preparation and analysis initiatives for artificial intelligence.
“Our CEO at Informatica often talks about the fact that as businesses outsource applications, architecture and many other aspects of the business, for many companies their only asset is their data. So this has to become a key issue for the executive team and the board of directors a high priority,” Scott said.
The growth of data management tools
The number of data management tools is constantly growing. Two-thirds (60%) of APAC leaders say they will need five or more data management tools to support priorities and manage data assets, up from the number of data owners who will need this many tools in 2023 (55%) increased.
Data governance and democratization
40% of regional data leaders see improving data and process governance as a top priority for their data strategy by 2024. APAC data leaders also place a high priority (67%) on achieving more democratization of data across the organization when using generative AI.
This has prompted vendors to offer governance services and tools. After Informatica acquired Privitar, it recently launched an integrated cloud data access management tool to help support the compliance management, sharing and use of data in various jurisdictions around the world.
SEE: By 2024, data governance will become the new focus in Australian organizational IT.
Informatica also offers a self-service data marketplace designed to “democratize” data access. Users can request and access data based on permissions. Data has data quality and relevance ratings and is tracked so that data stewards understand how the data is being used.
Infrastructure to meet data challenges
Informatica’s Richard Scott advises regional data leaders to build the right cloud architecture to support scale and focus on people, process and technology.
Start with the right cloud architecture
Scott said organizations should first ensure the soundness of their cloud architecture, as getting this right from the start will support future expansion efforts.
“When you scale and don’t have a data management architecture in place, you run into real trouble,” Scott said.
Scott adds that it’s also cheaper to go cloud-based from the start.
“Companies with multiple cloud contracts are paying a lot of money in entry and exit costs between clouds,” Scott said. “The wrong cloud architecture will not only result in an environment that cannot support generative AI, but it will also be very expensive.”
Informatica customer NRMA is one of Australia’s oldest membership organizations and is successfully processing more than 3,000 data sets. Organizations that strive to get the architecture right can take control of their data and have a significant impact on their data assets, Scott said.
Focus on people, processes and technology
The nature of the data challenge means organizations need to take a more holistic look at people, processes and technology. For data leaders in their organizations trying to solve problems as they arise, Scott said it can feel like “putting your fingers in a dam to stop a flood.”
“If you just plug every little hole in the dike by getting a new application or writing some code, what’s going to happen is you’re going to end up with a very fragmented environment that’s going to be very fragile. You need Focus on people, process and technology, and have a clear understanding of where you’re going; then you can bring in technology that’s highly integrated and gives you the ability to move data across your environment.”
