Hadoop adoption in the enterprise is growing steadily and, as the platform continues to rise in popularity, more companies are arriving on the front lines to find out how these technologies work.
In fact, a recent survey conducted over the subscribers of Database Trends and Applications found 55% of respondents currently use the platform and 44% have four or more clusters.
Alongside this momentum is a growing ecosystem of Hadoop-related projects including Apache Spark, Apache Hive, and Apache YARN.
To educate IT and business stakeholders about,how real companies are using them, the key challenges, and critical success factors, DBTA recently held a webinar featuring Russ Savage, application engineer at Cask; Kevin Petrie senior director and technology evangelist at Attunity; and Jamie Morgan, senior solutions architect at HPE Security - Data Security.
Users are faced with a variety of challenges, Savage explained. These include the complexity of projects, a widened skills gap, and a divergence of distributions and technologies. The lack of tools and reusable building blocks makes it difficult to take Hadoop into production.
According to Savage, Cask can help this problem as it 100% open source and highly extensible, supports all major Hadoop Distributions, integrates the latest big data technologies, including Kafka, YARN, Spark, Impala, HIVE, HIVE on Spark, HIVE on Tez.
Attunity can also address these issues, Petrie said, with data integration and big data management software that can accelerate data delivery and availability, automate data readiness for analytics, and optimize data management with intelligence.
After choosing the right tools, Morgan stressed that it’s imperative that users effectively secure their systems.
By applying format-preserving encryption users can allow wider access to their data lake, Morgan said. He offered tips to properly secure systems, including the need to protect data as close to the source as possible; protect once, decrypt rarely; utilize referential integrity across platforms; make sure to protect data at rest, in use, and in motion; and utilize applications that can use secure data without modification.
To view this webinar, go here.