Thank you for this. Framing data mesh as data’s application of microservices was helpful for me as someone relatively new to both. Overall, this has me thinking about what architectures are possible and ideal for my enterprise. Looking forward to your next post.
Really enjoyed this entry. Wondering if you'd be willing to elaborate on this point, which probably could be its own blog post.
When talking about source of truth datasets and the ability to fork the data for new R&D purposes, you also noted that "Data producers should be aware of changes to their dependencies when these promotions occur. They should be aware of the impact backward incompatible changes might have on data dependents as well."
My question is: What approaches have you seen work well for data producers to be aware of all the downstream dependencies when many independent R&D projects could depend on the same underlying data?
Doesn’t event sourcing address this issue? Having a persistent stream of raw events that consumers can subscribe to even after the fact? Having said this, I have found adopting event sourcing with existing systems is much harder, because people never really get over the fact that their old database is no longer the source of truth and just one particular read model. Easier to do for new systems.
Thank you for this. Framing data mesh as data’s application of microservices was helpful for me as someone relatively new to both. Overall, this has me thinking about what architectures are possible and ideal for my enterprise. Looking forward to your next post.
Really enjoyed this entry. Wondering if you'd be willing to elaborate on this point, which probably could be its own blog post.
When talking about source of truth datasets and the ability to fork the data for new R&D purposes, you also noted that "Data producers should be aware of changes to their dependencies when these promotions occur. They should be aware of the impact backward incompatible changes might have on data dependents as well."
My question is: What approaches have you seen work well for data producers to be aware of all the downstream dependencies when many independent R&D projects could depend on the same underlying data?
Doesn’t event sourcing address this issue? Having a persistent stream of raw events that consumers can subscribe to even after the fact? Having said this, I have found adopting event sourcing with existing systems is much harder, because people never really get over the fact that their old database is no longer the source of truth and just one particular read model. Easier to do for new systems.