Question 1

BN: Why has data modeling taken a back seat in recent years, and what is the cause of its resurgence?

Accepted Answer

SJ: Data modeling has always been one of the most critical steps in analytics projects, setting the groundwork to create databases, populate warehouses, manage data, and grant access to information in meaningful and governed ways. In recent years, the practice fell out of favor because it was 'too hard' and 'too slow'. Not surprisingly, the demand for speed often came at the expense of data quality. The general sentiment was that we needed to get data to our users, so we pushed the product out, but all we did was push the problem downstream to users. Strategic attention has returned to data modeling as businesses aim to extract measurable business value from their data. Data modeling is also helping companies cut compute costs as modeling is about publishing reusable, consumable artifacts, which data engineers and analysts can reuse and eliminate the need to repeat pipelines. In recent months, we've seen more emphasis on data modeling as it becomes even more critical with the increase in AI and machine learning use. Having trustworthy, governed data will result in accurate AI learnings and models, while data not in a suitable state or governed will skew AI and ML effectiveness.

Question 2

BN: What are some of the key challenges associated with modeling complex, unstructured data, and how do you overcome them?

Accepted Answer

SJ: As organizations consider rearchitecting their data warehouse, flexibility must remain a priority. Decommissioning and rearchitecting a data warehouse can be compared to the challenge of replumbing an entire neighborhood without causing any disruption to water service and supply -- it will require a flexible and adaptable model. Unstructured data comes in many forms, such as text, images, audio, and video. The challenges and solutions vary depending on the type of unstructured data. For instance, it's not uncommon for unstructured data to contain inconsistent, duplicate information. As a result, deciphering the context and extracting meaningful insights might become complicated and require additional data preprocessing methods to clean up and organize. Another challenge is processing unstructured data can be resource-intensive, and scalability could be a big issue. Using distributed processing frameworks and leveraging cloud platforms would be a way to address this particular challenge. Overall, exploratory data analysis, experimentation, and iterative modeling refinement are crucial for success. With a sound data modeling strategy, data can live up to its promises and deliver on the possibilities that come with proper practices, such as simplified, logical databases, reduced redundancies, minimal storage requirements, more efficient retrieval, and advanced analytics.

Question 3

BN: What role do you see machine learning and artificial intelligence playing in the future of data modeling?

Accepted Answer

SJ: Data modeling is a key piece of the AI puzzle. As we look at generative AI uses today, the output is only as good as the data it can access. The impact of data modeling on AI and machine learning is more pronounced, but both AI and machine learning will impact how data modeling and other data processes occur. The most significant role that AI/ML will play in the future of data modeling is speed and efficiency. The pattern recognition capabilities of AI/ML can automatically identify entities and the hidden relationships that exist between them based on historical data and, therefore, significantly speed up the data modeling process. AI/ML can potentially make data modeling accessible to individuals, particularly on the business side, unfamiliar with the data modeling practices but with more profound knowledge about the data itself. As a result, business users can build resilient and scalable data products.

Question 4

BN: How do you ensure that data models are aligned with business objectives and meet the needs of end-users?

Accepted Answer

SJ: Without a sustainable data architecture to deliver high-quality data, businesses will not reach the promise of predictive analytics, machine learning and AI or even be able to make truly data-driven decisions. And in today’s world of generative AI tools like ChatGPT, a business’s ability to successfully integrate and take advantage of these AI technologies to boost productivity and output and decrease spend and labor will distinguish leaders from laggards. One of the most common reasons for the failure of data projects is that the output produced doesn't meet the stated business needs. Since the data model is the foundation of a data analytics project, it's crucial to ensure that the built model aligns with the business objectives. There are several ways to achieve this, including the following:

Question 5

BN: How do you think data modeling will continue to evolve in the coming years?

Accepted Answer

SJ: Data modeling will evolve by integrating artificial intelligence and machine learning techniques for improved predictive capabilities. Automation and AutoML will streamline the modeling process, for instance, by identifying entities and detecting hidden relationships. As a result, self-service data modeling may become a reality where a businessperson with a solid understanding of the data but needs to gain data modeling experience can actively participate in the process.

Question 6

BN: What are some of the key factors to consider when optimizing data models for performance and scalability?

Accepted Answer

SJ: In general, there is yet to be a universally optimized data model. Instead, a data model is optimized based on the specific workload it is designed to support. For example, a highly normalized data model is well-suited for an operational database prioritizing transactional integrity and write performance. On the other hand, a denormalized model like a star schema is more suitable for an analytics database where read performance takes precedence. The optimization of a data model depends on the intended use case and the specific requirements of the workload it will handle. Choosing appropriate data storage technologies, such as relational databases, NoSQL databases, or data lakes, depends on data volume, data structure, query patterns, and scalability requirements. Efficient indexing, partitioning, and caching strategies can optimize data access and retrieval.

Question 7

BN: How can organizations stay up to date with the latest trends and best practices in data modeling?

Accepted Answer

SJ: Staying current with the latest trends and best practices in data modeling is crucial for organizations to ensure they are leveraging the most effective techniques. Here are some strategies that organizations can adopt to stay current: Image credit: realinemedia/depositphotos.com

data modeling

The return of data modeling -- this time it's strategic [Q&A]

Recent Headlines

Threats improve to slip past firewalls and filters

Paranoia rules -- how automation can enable better detection and response [Q&A]

Google reveals Year in Search 2025

Apple announces more changes at the top

Microsoft is preparing to give the Windows 11 Run dialog a much-needed makeover

Google shares its favorite Chrome extensions of 2025 as AI takes over the browser

Apple unveils the 2025 App Store Awards winners -- is your top pick on the list?

Most Commented Stories

Spotify Wrapped 2025 is here, and this time it’s interactive and competitive

Crucial brand to disappear as Micron shifts focus to AI data center memory

Open-source BitTorrent app FrostWire 7 launches with rebuilt engine and zero ads

Google shares its favorite Chrome extensions of 2025 as AI takes over the browser

Opera brings upgraded Google AI to its browsers

Google has dropped its antitrust complaint against Microsoft in the EU

Raspberry Pi launches 1GB Raspberry Pi 5 at $45, raises the price of other models

Desktop web traffic overtakes mobile for the first time in 5 years

NEWS