<p>Company values are the core of an organization, and these values should be shared and practiced by every member. Canner&#39;s company values consist of five core values: <strong>strategic thinking, collaboration, commitment, passion, and innovation</strong>.</p>
<h2 id="1-strategic-thinking">1. Strategic thinking</h2>
<p>Strategic thinking is the ability to think independently and strategically, to analyze the root cause of a problem and develop solutions. Team members should make decisions based on long-term goals, rather than pursuing short-term benefits.</p>
<h2 id="2-collaboration">2. Collaboration</h2>
<p>Collaboration involves respecting others&#39; opinions and seeking consensus during discussions. Team members should also help each other complete their work, even under high-pressure situations, while maintaining a calm and clear mindset.</p>
<h2 id="3-commitment">3. Commitment</h2>
<p>Commitment means that every team member should take responsibility for the company&#39;s performance. They should use smart methods to solve problems, and when mistakes happen, team members should take responsibility together and make quick corrections.</p>
<h2 id="4-passion">4. Passion</h2>
<p>Passion means that every team member should have a passionate and dedicated attitude towards their work. Team members should identify with the company&#39;s vision and move towards goals with enthusiasm and the team.</p>
<h2 id="5-innovation">5. Innovation</h2>
<p>Innovation means challenging traditional methods and exploring innovative solutions by rethinking the fundamental problems. Team members should be bold in making assumptions and accepting failures in order to create better solutions.</p>
<p>Through these five core values, we believe that every member of Canner can achieve the impossible together. </p>
<blockquote>
<p>&quot;I&#39;m a Canner, I&#39;m possible.&quot;</p>
</blockquote>


Company values are the core of an organization, and these values should be shared and practiced by every member. Canner's company values consist of five core values: strategic thinking, collaboration, commitment, passion, and innovation.

Canner's Company Values

Canner has been invited to participate in the "Manufacturing and Electronics Industry Transformation from 'Worker Intelligence' to 'Artificial Intelligence'" forum at the Galaxy Summit 2021 AI conference.

【2021 Galaxy】AI Empowerment

<p>We expect the Canner team to adhere to the following core principles when it comes to collaboration. - <em>Character is what you do, when no one else is watching</em></p>
<h2 id="1-commit-and-focus">1. Commit and focus</h2>
<p><em>Focus and commitment are what make everything different.</em></p>
<h2 id="2-independent-thinking-and-effective-communication">2. Independent thinking, and effective communication</h2>
<p><em>Your unique talent and thoughts are what make you different, make a difference.</em> </p>
<p>We need to create environments in which everyone feels welcomed, fairly treated, and fully supported to do their best. In short, it&#39;s about prioritizing mutual respect.</p>
<h2 id="3-proactive-and-action">3. Proactive, and action</h2>
<p><em>Do what matters, now.</em> </p>
<h2 id="4-working-backward">4. Working backward</h2>
<p><em>Set your end goals first, then weight your decisions.</em></p>
<h2 id="5-quality-with-speed">5. Quality with Speed</h2>
<p><em>Deliver with quality makes things count.</em> </p>
<h2 id="6-customers-first">6. Customers first</h2>
<p><em>Happy customers are why a company exists.</em></p>
<h2 id="7-team-work">7. Team work</h2>
<p><em>Great things in business are never done by one person. They&#39;re done by a team of people. — Steve Jobs</em> </p>


Canner's Teamwork Guidelines

<p>AI has been the raging buzzword globally and across industries over the past few years. Whilst many have yet to take that bold leap, many others have tried, paused, become disillusioned, and only a few have managed to cross that chasm. </p>
<p>“Where exactly am I in that cycle? What have others who have succeeded done? What do I need to make this a successful conquest? Where do I go next?” These questions arise in every board room and meeting we’ve had with large and small enterprises in their quest to transform and optimize their business. </p>
<p>This year, Hive Ventures decided to answer these questions, by embarking on a nationwide survey, to help business decision-makers and solutions providers alike, unveil the mystery behind the current state of enterprise AI adoption in Taiwan. </p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/m6_Sq19EdZs" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

<p>Get your Free copy today! 👉 <a href="https://www.hiveventures.io/sotea">https://www.hiveventures.io/sotea</a></p>
<h3 id="canner-enterprise---data-access-solution-for-enterprises">Canner Enterprise - Data Access Solution for Enterprises.</h3>
<ul>
<li>【三立新聞】不只半導體領先！台灣企業部署AI人才　成亞洲領頭羊
<a href="https://www.setn.com/News.aspx?NewsID=915381">https://www.setn.com/News.aspx?NewsID=915381</a></li>
<li>【電子時報】蜂行資本：台灣AI應用面3年內有望趕上美國
<a href="https://www.digitimes.com.tw/iot/article.asp?cat=158&amp;cat1=20&amp;cat2=10&amp;id=0000606781_FBD863R37X5P712OQHIPL">https://www.digitimes.com.tw/iot/article.asp?cat=158&amp;cat1=20&amp;cat2=10&amp;id=0000606781_FBD863R37X5P712OQHIPL</a></li>
<li>【聯合報】台灣八成企業積極導入AI 但門檻是「缺有經驗人才」
<a href="https://money.udn.com/money/story/5612/5341905">https://money.udn.com/money/story/5612/5341905</a></li>
<li>【數位時代】8成台灣企業已經導入AI，一張圖看懂進度到哪裡！還面臨了哪些挑戰？
<a href="https://www.bnext.com.tw/article/61947/tw-ai-trend-2021">https://www.bnext.com.tw/article/61947/tw-ai-trend-2021</a></li>
<li>【中時】台灣已有八成以上企業嘗試導入AI 企業渴望挖掘有經驗人才
<a href="https://turnnewsapp.com/ct-featured/223860.html">https://turnnewsapp.com/ct-featured/223860.html</a></li>
<li>【自由時報】調查：8成企業嘗試導入AI 台美AI應用差距3年內有望追近
<a href="https://ec.ltn.com.tw/article/breakingnews/3478649?utm_medium=APP&amp;utm_campaign=SHARE&amp;utm_medium=APP">https://ec.ltn.com.tw/article/breakingnews/3478649?utm_medium=APP&amp;utm_campaign=SHARE&amp;utm_medium=APP</a></li>
</ul>
<div class="gallery">
    <img src="/static/images/ws-post/2021_03_24_galaxy_summit/pic_1.png"/>
    <img src="/static/images/ws-post/2021_03_24_galaxy_summit/pic_2.png"/>
    <img src="/static/images/ws-post/2021_03_24_galaxy_summit/pic_3.png"/>
    <img src="/static/images/ws-post/2021_03_24_galaxy_summit/pic_4.png"/>
    <img src="/static/images/ws-post/2021_03_24_galaxy_summit/pic_5.png"/>
    <img src="/static/images/ws-post/2021_03_24_galaxy_summit/pic_6.png"/>
    <img src="/static/images/ws-post/2021_03_24_galaxy_summit/pic_7.png"/>
</div>

This year, Hive Ventures decided to answer these questions, by embarking on a nationwide survey, to help business decision-makers and solutions providers alike, unveil the mystery behind the current state of enterprise AI adoption in Taiwan.

State of Taiwan Enterprise AI

<h2 id="1-ownership-mentality">1. Ownership Mentality</h2>
<p>Leaders have long-term goals and will not sacrifice long-term value for short-term performance. They think from the perspective of the owner. Their every move represents not only their own team, but also the entire company. They never say &quot;that&#39;s not my job&quot;.</p>
<h2 id="2-earn-trust">2. Earn Trust</h2>
<p>Leaders listen attentively, communicate openly, and respect others. Leaders are brave enough to be self-critical, even if doing so embarrasses or embarrasses them. Not believing themselves or their teams to be always right, leaders hold themselves and their teams to the highest standards.</p>
<h2 id="3-have-backbone-disagree-and-commit">3. Have Backbone; Disagree and Commit</h2>
<p>Leaders should question decisions they disagree with without condescension, even if doing so makes people uncomfortable or exhausting. Leaders should be strong-willed and not easily wavered. They will not easily compromise in order to maintain social cohesion. When a decision is made, they move forward.</p>
<h2 id="4-think-big-start-small-and-move-fast">4. Think Big, Start Small, and Move Fast</h2>
<p>Leaders should formulate and clarify the overall policy and implementation strategy, taking into account bold innovation and inspiration team. Consider issues from different angles, start with the most important things in stages, and execute quickly and accurately.</p>
<h2 id="5-deliver-results">5. Deliver Results</h2>
<p>Leaders will pay attention to the key factors that determine the success or failure of things, and can ensure quality and timely completion. Even if they encounter setbacks, leaders still face them bravely and never give up easily.</p>


Five Leadership Guidelines followed by the Canner team. These principles guide the team towards long-term success by encouraging strategic thinking, open communication, decisive action, and a commitment to quality and results.

Canner's Leadership Guidelines

<p>Nowadays, enterprise data is very complex siloed, and scattered across many different data sources and systems. Being able to retrieve and collecting the holistic data overview is becoming a huge task.  Companies need to have a new approach to solve and apply to their data ecosystem, that is “Data Fabric”.</p>
<h2 id="what-is-a-data-fabric">What is a Data Fabric?</h2>
<p>Conceptually, Data Fabric is an abstraction layer across all the data sources, to empower data teams to collect data information across different sources.  Data Fabric is a distributed data environment that connects all data together so that it can be seamlessly accessed by data management services and utilized by end-users, or by applications, and more importantly, data is not physically moved, it is connected by a virtual layer.</p>
<p>Here’s the Data Fabric definition from Gartner.</p>
<p> <img src="/static/images/ws-post/2021_05_22_intro_data_fabric/gartner.png" alt="Gartner report - Data Fabric Architecture"></p>
<p>Source From <a href="https://www.gartner.com/smarterwithgartner/data-fabric-architecture-is-key-to-modernizing-data-management-and-integration/">Gartner Data Fabric Architecture is Key to Modernizing Data Management and Integration</a></p>
<h2 id="key-features">Key features</h2>
<ul>
<li><p><em><strong><strong>Seamlessly integrated with your existing data architecture.</strong></strong></em></p>
<p>The Data Fabric must capable of integrating existing data architecture and data sources, by simply connect to the sources. Data sources may be public clouds or on-premises, Data Fabric technology needs to provide a single layer for data access and delivery, this way data silos can be connected through a virtual layer.</p>
</li>
<li><p><em><strong><strong>Robust data outputs and destinations</strong></strong></em></p>
<p>Through the Data Fabric complex data transformation and optimization, end-users are able to connect to different outputs such as AI, BI, and analytics software.</p>
</li>
<li><p><em><strong><strong>Metadata catalog and management</strong></strong></em></p>
<p>In Data Fabric software one of the most important features is metadata catalog, which allows data to be searchable, downloadable and all business users can easily access and share between different units and departments.</p>
</li>
<li><p><em><strong><strong>Flexible data modeling and transformation</strong></strong></em></p>
<p>Using Data Fabric it is also very easy for users to model and transform their data into different dimensions, so different business users are able to inspect and review the data in different aspects and scenarios.</p>
</li>
<li><p><em><strong><strong>Data sharing &amp; protection</strong></strong></em></p>
<p>Data sharing within an organization and the cross-functional departments is a huge challenge, most enterprises will encounter huge data management bottlenecks while they are scaling their data pipelines.  Also, data protection between each user will need a fine-granular permission layer (schema, table, column-level of access control), a built-in user/role-based permission system, and versioning.</p>
</li>
</ul>
<h2 id="canner">Canner</h2>
<p>With Canner data access solution, enterprises can connect data silos virtually, and accelerate business data analysis, while ensure data privacy, with data policies.</p>
<ul>
<li>Connect with data sources in clicks.</li>
<li>Optimization for data outputs</li>
<li>Metadata management</li>
<li>Data transformation</li>
<li>Virtual data mart for data sharing &amp; protection.</li>
</ul>


In this article, we would like to introduce a new approach to your business called "Data Fabric"...

Introduction to Data Fabric

<h2 id="interview-with-digitimes">Interview with Digitimes</h2>
<p>In 2011, the World Economic Forum (WEF) pointed out in its Global Risk Report that data was going to become the most valuable resource of the 21st century. According to a report from the International Data Corporation (IDC), global data volume is projected to grow to 163 ZB in 2025, showing a tenfold increase from the 16.1 ZB in 2016. The massive amount of data in combination with the rise of big data and data surveying tools mean that uncovering business opportunities from enormous data has already become an essential task for businesses looking to increase their competitiveness. Consequently, this has drawn great focus to data access, otherwise known as data management platforms. </p>
<p>In response to increasing enterprise demand for big data access, many software companies have begun to offer solutions targeting this market, with the most notable among them being the Taiwanese data company Canner Inc. The company was founded in 2016 with an initial focus on developing content management systems but was able to secure investment from the National Development Fund, Hive Ventures, and SparkLabs Taipei in 2018 to develop its Canner Data Access Solution, which was created in response to growing enterprise data surveying demand. The solution currently targets three industries – finance, manufacturing, and retail. </p>
<p>Furthermore, the solution can also compete with foreign options in terms of both price and time by offering much more competitive pricing and allowing projects to be completed within a span of days.</p>
<h2 id="quick-installation-affordable-pricing-and-support-for-multiple-deployment-modes-and-data-connection-sources">Quick Installation, Affordable Pricing, and Support for Multiple Deployment Modes and Data Connection Sources</h2>
<p>Unlike other solutions, Canner only requires a few minutes to install and can easily adapt to various database types, file formats, and cloud/local limitations. The solution can quickly integrate data from different databases and connect isolated data lakes to integrate, convert, synergize, and extricate the truly valuable parts from a sea of data, effectively increasing business competitiveness. Considering the rapid growth of data nowadays, the software also has automated optimization and extensible computation to bypass the lack of relevant manpower in certain businesses, so that even medium-sized businesses can construct their own data platform.</p>
<p>Howard noted that Canner is not only capable of quickly connecting various databases and data stores, but can also be linked to business intelligence (BI) and artificial intelligence (AI) software as well as various data applications to ease program development efforts. Additionally, database administrators can assign necessary access rights based on the jobs and needs of staff in various departments so that every staff member will always have access to data they need, thereby creating the greatest value from data.</p>
<p>Take manufacturing for example, Canner can help businesses rapidly integrate factory information, M&amp;A data, logistic data, and information from both upstream and downstream partners to shorten the time it takes for the product to arrive on the market. For marketing purposes, the solution can be applied to cross-screen, cross-platform, OMO, and O2O data integration. It is worth mentioning that Canner offers flexible installation options to allow businesses to deploy the software in a designated environment according to their preferences and needs. Canner Cloud in particular can be installed within the private cloud of a business to prevent potential data leaks and thereby meet information security and operation demands.</p>
<h2 id="mentorship-from-tta-partner-accelerator-program-helps-accelerate-product-development-cycle">Mentorship from TTA Partner Accelerator Program Helps Accelerate Product Development Cycle</h2>
<p>Since Canner Inc. entered the data access market in 2018, the technical prowess of the company has been repeatedly tested and reaffirmed. The Canner virtual Data Access Solution has also gained presence in various industries and increasing renown in Taiwan. This is not only the result of hard work from the team behind the software, but also has a lot to do with capital and technical assistance from the National Development Fund, Hive Ventures, and SparkLabs Taipei. </p>
<p>Howard indicated that one of the key reasons that Canner Inc. products were able to hit the market so quickly is because of assistance from SparkLabs Taipei, an international accelerator program introduced to Taiwan by Taiwan Tech Arena (TTA). The program offers mentorship and advice to help him and his team reduce learning time and overcome challenges. Howard believes that Taiwan is in an excellent position to develop its information technology industry. Canner Inc. will continue to invest in this industry to provide the best virtual data center solution for global businesses. </p>
<hr>


Digitimes Interview with our co-founders - We are honored to be invited to share our B2B enterprise experience with Digitimes.

【TTA Magazine】Data Access Solution for Enterprises

<p>Canner, a data access platform, has announced that its data virtualization offering now fully supports over 20 different data sources. The platform&#39;s new features allow users to easily integrate data from a wide range of sources, including data warehouses, databases, and data lakes, among others.</p>
<p>The data consumption interface is fully compatible with the PostgreSQL wire protocol, which means that it can support any data applications that support PostgreSQL connectors. This makes it possible for businesses to easily access and use data from a variety of sources, regardless of the specific tools or applications they are using.</p>
<p>&quot;We&#39;re thrilled to announce these new features for Canner Enterprise,&quot; said Howard Chi, CEO of Canner Enterprise. &quot;Our goal has always been to make it as easy as possible for businesses to access and use their data, and these new features are a big step towards achieving that goal. We&#39;re confident that our customers will find these new capabilities to be incredibly valuable.&quot;</p>
<img src="/static/images/ws-post/2022_01_16_canner_dv/founder.png"/>

<p>Canner Enterprise&#39;s data virtualization offering is designed to help businesses streamline their data management processes, reducing the time and resources required to integrate and use data from multiple sources. With these new features, businesses can more easily access and use data from a wide range of sources, making it easier to gain insights and make informed decisions.</p>
<p>For more information about Canner Enterprise&#39;s data virtualization offering, visit the company&#39;s website or contact a member of the Canner Enterprise team today at <a href="mailto:&#x73;&#97;&#108;&#x65;&#x73;&#x40;&#x63;&#x61;&#110;&#x6e;&#101;&#x72;&#100;&#97;&#x74;&#97;&#x2e;&#99;&#111;&#x6d;">&#x73;&#97;&#108;&#x65;&#x73;&#x40;&#x63;&#x61;&#110;&#x6e;&#101;&#x72;&#100;&#97;&#x74;&#97;&#x2e;&#99;&#111;&#x6d;</a>.</p>


Canner Enterprise's data virtualization platform now supports over 20 data sources and is fully compatible with the PostgreSQL wire protocol, streamlining businesses' data management processes.

Canner Enterprise Data Virtualization fully compatible with PosgreSQL wire protocol interface

<p>Canner, a Taiwan-based data access solution provider, announced Tuesday that it has raised a total of $3.5 million in a Series Pre-A funding round.</p>
<p>Led by Taiwania Capital, the funding round was also participated in by global venture capital firm Hive Ventures and SparkLabs Taipei, Canner said in a statement.</p>
<img src="/static/images/ws-post/2022_03_22_canner_pre_a/visual.png"/>

<p>Canner will use this funding to accelerate product development, expand local and international marketing efforts, and grow its internal teams.</p>
<p>Founded in 2018, Canner aims to empower businesses to efficiently convert data into business value by connecting data silos and transforming business-facing datasets into application-ready dataset APIs with a universal data access interface. With Canner’s data access technology, users can work with datasets without moving or duplicating data between data sources, simplifying the process of building next-generation data applications on top of cloud data warehouses through a universal layer for APIs, access control, data literacy and optimization from diverse data silos.</p>
<p>“Enterprises used to struggle with data silo issues. Even though most data are now centralized in a data warehouse, balancing business innovation and data security remains a big hurdle for organizations due to the complexity and sensitivity around working with data. That is where Canner comes in,</p>
<p>“With Canner’s data access technology, enterprises can virtually enhance their connectivity to data silos and guarantee data security and privacy when collaborating datasets between business units within a company,” said Howard Chi, Co-Founder and Chief Executive Officer of Canner.</p>
<p>Canner’s data access solution can be quickly installed in any cloud – public, private, hybrid and otherwise. It provides multi-format output optimization for data applications, as well as multi-layer data access control and authorization. This allows it to empower organizations to self-serve datasets, achieve data democratization and accelerate innovation, thereby greatly improving enterprise data-driven efficiency.</p>
<p>“The limitations in data availability and accessibility at scale are significant obstacles to enterprise digital transformation. Taiwania believes in Canner’s vision to democratize the emerging data access ecosystem by embracing open-source framework that promises to seamlessly integrate next generation business intelligence applications with analytic tools,” said Cheng Wu, General Partner at Taiwania Capital. An industry veteran, Cheng Wu has founded several high-profile companies with total market value of $6 billion and have led several successful exits.</p>
<p>With its existing support for private and mainstream cloud vendors such as AWS, GCP and Azure, Canner plans to expand beyond its robust local customer base in Taiwan by embracing open-source frameworks and providing Canner cloud Software-as-a-Service (SaaS) services that are available to global customers.</p>
<p>This year, the company also aims to partner with global cloud data warehouse providers such as Redshift, Snowflake, and BigQuery, as well as major solution integrator partners, to accelerate cloud data warehouse and data lake usages across organizations.</p>
<p>“We previously invested in Canner’s seed funding round in 2019 and our participation in this round signifies our continued confidence in their business model. There is no question that data will be the building blocks of future enterprise success, and it is critical that businesses find more efficient ways to access and analyze their insights to enable more agile business transformation. With successful entrepreneurs like Cheng Wu now on the Board of Advisors, we look forward to seeing Canner lead the charge in the data solutions space,” said Yan Lee, Founding Partner of Hive Ventures.</p>


Led by Taiwania Capital, the funding round was also participated in by global venture capital firm Hive Ventures and SparkLabs Taipei.

Canner raises $3.5M Series Pre-A funding led by Taiwania Capital

<p>Canner, a data access platform, is proud to announce that Cheng Wu has joined the company as an advisor. Cheng is a highly accomplished entrepreneur with a proven track record in the technology industry, having founded several successful companies and currently serving as the General Partner of Tech Fund at Taiwania Capital.</p>
<p>Cheng&#39;s entrepreneurship journey began in 1995 when he founded Arris Networks, a startup that developed high-density internet access products that were acquired by Cascade Communications for $217 million in May 1996. In 1997, he founded ArrowPoint Communications and served as CEO, leading the company to a successful IPO. ArrowPoint was acquired by Cisco Systems for $5.7 billion in 2000, and Cheng held various executive positions, including Group VP.</p>
<p>In 2002, Cheng founded Acopia Networks with $65 million, which was later acquired by F5 Networks for $210 million in 2007. That same year, he founded Azuki Systems, specializing in broadband wireless multimedia data transfer software, with a startup capital of $6 million. Azuki Systems was subsequently acquired by Ericsson for $100 million in February 2014.</p>
<p>With his vast experience and expertise in the technology industry, Cheng will provide valuable insights and guidance to Canner. His extensive background in building successful companies will be instrumental in driving Canner&#39;s growth and success in the future.</p>
<p>Canner is thrilled to welcome Cheng Wu as an advisor and looks forward to a fruitful partnership with him.</p>


Cheng Wu, a highly accomplished entrepreneur with a successful track record in the technology industry, has joined Canner as an advisor. With his extensive experience and expertise, Cheng will provide valuable insights and guidance to drive Canner's growth and success in the future.

Cheng Wu Joins Canner as Advisor to Drive Growth and Success

<p>Canner, the data access company, is excited to announce the launch of its new deployment support on AWS, GCP, and Azure, on single VM or Kubernetes environments. This new feature is a significant step towards making Canner&#39;s platform more versatile and flexible for businesses and organizations that require a highly customizable data access solution.</p>
<p>Canner&#39;s platform enables businesses and organizations to collect, manage, and analyze data efficiently, and this new deployment support makes it even easier for users to deploy and manage Canner on their preferred cloud platform. Users can now choose between a single VM deployment or Kubernetes deployment to suit their specific requirements.</p>
<p>&quot;We are thrilled to offer our users the ability to deploy and manage Canner on their preferred cloud platform,&quot; said the Canner team. &quot;Our platform is designed to be highly flexible and customizable, and this new deployment support is a testament to that.&quot;</p>
<img src="/static/images/ws-post/2022_05_03_cloud_deployment/founders.png"/>

<p>The new deployment support provides users with a comprehensive guide to deploying Canner on AWS, GCP, and Azure, on single VM or Kubernetes environments. It covers all aspects of the deployment process, including how to set up the environment, how to configure the necessary settings, and how to manage the deployment on an ongoing basis.</p>
<p>To learn more about Canner and its new deployment support, please visit <a href="https://cannerdata.com/">https://cannerdata.com/</a>. The Canner team is always available to provide support and answer any questions that users may have about the platform or the deployment process. With this new deployment support, businesses and organizations can deploy and manage Canner with ease on their preferred cloud platform.</p>


Canner, the data access company, has launched new deployment support for AWS, GCP, and Azure on single VM or Kubernetes environments. The new feature provides businesses and organizations with flexible deployment options for Canner's platform, making it easier to manage and analyze data efficiently.

Canner Launches New Deployment Support for Cloud Platforms

<h2 id="introduction">Introduction</h2>
<p>The growth of data innovation has exploded in recent years, mainly due to the thriving cloud data warehouse communities such as Snowflake, Redshift, and BigQuery, with their primary focus on SQL users and business intelligence use cases. Lakehouse architecture brings analytics closer to data lakes, enabling heterogeneous and distributed data processing engines to ingest sources, including diverse workloads such as data science, machine learning capabilities, and near real-time analytics enablement. It has also spawned thriving innovations in integrated data services that automate and unify data modeling, transformation, and metrics, such as dbt and LookML. Collectively, these tools lay the foundation upon which next-generation operational and analytical data applications can be constructed for various data consumers of different personas.</p>
<h2 id="the-data-access-challenges">The Data Access Challenges</h2>
<p>First of all, let&#39;s look at the ultimate goal for data access in a company:</p>
<blockquote>
<p>Allow data consumers to find and access their data, fast and simple.</p>
</blockquote>
<p>Let&#39;s look at a company&#39;s current data access workflow; here&#39;s what it looks like for a data consumer requesting and accessing data.</p>
<ol>
<li>Data consumers need to find who owns the data.</li>
<li>Request access from the data owners; then, they must filter or mask certain rows and columns to specific groups or users before exposing them to usage.</li>
<li>Based on different data applications/tools, such as using Excel, BI, AI, or RESTful API, data owners need to evaluate the best data access method.</li>
<li>Last is automation. Periodically, update and deliver to end applications and ensure they are secure and auditable.</li>
</ol>
<p>Data access in an enterprise encounters several challenges.</p>
<p><img src="/static/images/ws-post/2022_06_27_enterprise_vision/without-canner-en.png" alt="Without Canner"></p>
<p>From left to right, the data heterogeneity needs to homogenize in the metadata and logical level; data authorization is a sophisticated access control and authorization for domain-specific datasets and their associated data applications; empower datasets with semantic meanings through the process of data productization; Finally, based on different data consumers&#39; persona provide endpoints.</p>
<blockquote>
<p>A Universal Semantic Layer must solve data heterogeneity, usability, and authorization while enabling consistency and scalability across different data applications.</p>
</blockquote>
<h2 id="the-universal-semantic-layer">The Universal Semantic Layer</h2>
<p>As companies scale, they can quickly become overwhelmed with data requests, leading to a backlog of demands that can take days or even weeks to resolve. The issue lies in the data access workflow. To overcome this bottleneck, a secure, efficient, and intelligent Universal Semantic Layer is essential, enabling data consumers to access data independently, avoiding delays and misalignment, while ensuring that data owners can authorize and audit datasets, guaranteeing that only the right individuals have access to the appropriate data.</p>
<blockquote>
<p>We need a Universal Semantic Layer that is secure, efficient, and intelligent.</p>
</blockquote>
<p>Ultimately, achieve equilibrium between <em><strong>data, people, and applications</strong></em>.</p>
<img alt="balance" src="/static/images/ws-post/2022_06_27_enterprise_vision/balance-en.png"/>

<h2 id="four-key-designs">Four key designs</h2>
<p>Canner Enterprise centralizes governance and control across data applications, allowing for unified access to diverse data sources. We offer a more efficient and effective way to manage and govern data, reducing complexity and risk while providing greater transparency and control.</p>
<h3 id="1-data-virtualization">1. Data Virtualization</h3>
<p>Universal Semantic Layer is collaborative and distributed in nature, with each silo or data source independently scalable or together as an aggregate.</p>
<h3 id="2-data-product-management">2. Data Product Management</h3>
<p>Transform data models to domain-oriented datasets; Domain-oriented datasets owned by data owners can be shared and governed by open APIs, with the flexibility of interchangeable metadata and access rules, let data speak your business language.</p>
<h3 id="3-data-security">3. Data Security</h3>
<p>Consistent data authorization framework from sources to data applications and integrated with existing Identity and Access Management (IAM). Make data authorization consistent across data sources, IAM, and data applications.</p>
<h3 id="4-data-serving">4. Data Serving</h3>
<p>Data consumers can generate Queries and APIs with intent and contextual settings, applied to the corresponding datasets via intent declaration, and deliver them to target consumers where final analytics are performed and displayed.</p>
<p><img src="/static/images/ws-post/2022_06_27_enterprise_vision/with-canner-en.png" alt="Canner structure"></p>
<h2 id="the-result">The Result</h2>
<p>Enterprises can significantly eliminate data complexity, communication, and productivity through the Universal Semantic Layer.</p>
<ol>
<li><em><strong>Data access from days to minutes:</strong></em> Reduce 60% of data integration cost with up-to-date data delivery.</li>
<li><em><strong>Reduce duplicate datasets:</strong></em> Create masked and filtered datasets without physically moving data.</li>
<li><em><strong>Achieve self-service analytics:</strong></em> Improve data productivity across analytical and operational data applications and tools.</li>
</ol>


In this article, we will share how a universal semantic layer can solve common enterprise data access challenges.

How Universal Semantic Layer solve data access challenges?

<p>In the modern manufacturing industry, data is essential for optimizing operations, improving product quality, and enhancing customer satisfaction. However, with the ever-increasing volume and variety of data, accessing and managing it has become a significant challenge for manufacturers. Fortunately, Canner Enterprise, the data access platform, is here to help manufacturers solve their data-related problems using data virtualization technology.</p>
<p>Canner Enterprise is a powerful data access platform that provides manufacturers with a unified view of their data from multiple sources, regardless of its format, location, or structure. With Canner Enterprise, manufacturers can easily access and integrate data from various systems, such as ERP, CRM, MES, among others.</p>
<p>Manufacturers face numerous challenges when it comes to data management. For example, in a highly competitive market, manufacturers must optimize their operations to reduce costs and improve productivity continually. Canner Enterprise can help manufacturers access and analyze real-time data from various production systems, enabling them to identify inefficiencies, bottlenecks, and other issues that impact productivity.</p>
<h2 id="todays-data-challenge">Today’s data challenge</h2>
<p>Manufacturing without data virtualization can be challenging and inefficient, as manufacturers face several pain points related to data management and access. Here are some of the most significant pain points of manufacturing without data virtualization:</p>
<ol>
<li><strong>Data silos</strong>: Manufacturing systems generate vast amounts of data, which often gets stored in isolated silos, making it challenging to access and analyze. This can lead to a fragmented view of operations, and manufacturers may miss valuable insights and opportunities for optimization.</li>
<li><strong>Complex integrations</strong>: Integrating data from various systems can be a complex and time-consuming process, especially if the systems use different formats or protocols. Manufacturers may need to develop custom integration solutions or rely on manual data transfers, which can be error-prone and inefficient.</li>
<li><strong>Data duplication</strong>: In many cases, manufacturers need to replicate data across multiple systems to ensure all teams have access to the necessary information. This can lead to data duplication, inconsistencies, and errors, making it difficult to trust and rely on the data.</li>
<li><strong>Limited access</strong>: Data may be inaccessible to some teams or stakeholders, leading to siloed decision-making and missed opportunities for optimization. In some cases, manufacturers may need to provide physical access to data centers or require specialized skills to access the data, limiting its usability.</li>
<li><strong>Slow data retrieval</strong>: Without data virtualization, retrieving data from various systems can be slow and time-consuming. Manufacturers may need to perform multiple queries or wait for manual data transfers, slowing down decision-making and hindering operational efficiency.</li>
</ol>
<h2 id="scenarios-of-using-data-virtualization-in-manufacturing">Scenarios of using Data Virtualization in manufacturing:</h2>
<p>Data virtualization is a technology that allows businesses to access and integrate data from multiple sources without having to physically move or replicate the data. In the manufacturing industry, data virtualization can be used in a variety of ways to improve data access, integration, and analysis. Here are some scenarios of using data virtualization in manufacturing:</p>
<ol>
<li><strong>Supply chain management</strong>: Manufacturers can use data virtualization to integrate data from various sources such as suppliers, logistics providers, and internal systems to gain a more comprehensive view of their supply chain. By virtualizing data, they can quickly and easily access data in real-time, perform simulations and analyses, and make more informed decisions.</li>
<li><strong>Production planning</strong>: Manufacturers can use data virtualization to access and integrate data from various production systems, including machine sensors, shop floor systems, and quality control systems. By virtualizing data, they can quickly analyze production data, identify patterns, and optimize production processes.</li>
<li><strong>Quality control</strong>: Manufacturers can use data virtualization to integrate data from various quality control systems, including production systems, supplier systems, and customer feedback systems. By virtualizing data, they can quickly identify quality issues, track quality metrics, and make improvements to their quality control processes.</li>
<li><strong>Asset management</strong>: Manufacturers can use data virtualization to integrate data from various asset management systems, including machine sensors, maintenance systems, and inventory systems. By virtualizing data, they can quickly analyze asset performance, track maintenance requirements, and optimize asset utilization.</li>
<li><strong>Sales and marketing</strong>: Manufacturers can use data virtualization to integrate data from various sales and marketing systems, including customer databases, market research systems, and social media feeds. By virtualizing data, they can quickly analyze customer behavior, identify trends, and improve sales and marketing strategies.</li>
</ol>


We discuss the challenges that manufacturers face when managing and accessing data, and how data virtualization technology can help. The platform provides a unified view of data from multiple sources, allowing manufacturers to access and integrate data easily.

Data Virtualization in Manufacturing

<p>In today&#39;s world, data is the backbone of every business, and the finance sector is no exception. Financial institutions like banks, insurance companies, and securities firms rely heavily on data to make strategic decisions, manage risks, and provide excellent customer service. However, with the exponential growth of data, managing, accessing, and analyzing it has become a daunting task.</p>
<p>Fortunately, there is a solution that can simplify data management and access for financial institutions - data virtualization. And Canner Enterprise, the data access layer, is at the forefront of this technological revolution.</p>
<h2 id="canner-enterprise--data-access-platform">Canner Enterprise — Data Access Platform</h2>
<p>Canner Enterprise is a powerful data access platform, that leverage data virtualization technology that allows financial institutions to access and manage data from multiple sources, regardless of the data&#39;s format, location, or structure. This means that financial institutions can easily access data from various systems, such as core banking, CRM, risk management, and compliance systems.</p>
<p>With Canner Enterprise, financial institutions can solve many financial problems, such as financial holding, insurance, bank, securities, and more. For instance, in financial holding companies, data is scattered across various business units and systems, making it challenging to gain a complete view of the organization&#39;s operations. Canner Enterprise can provide a unified view of the organization&#39;s data, enabling managers to make better decisions and identify potential risks and opportunities.</p>
<p>In the insurance sector, data is crucial for risk assessment, underwriting, and claims management. Canner Enterprise can provide insurance companies with a 360-degree view of their customers, policies, and claims, enabling them to provide personalized services and streamline claims processing.</p>
<p>For banks, compliance is a critical issue. Canner Enterprise can help banks access and manage data from various sources, such as anti-money laundering systems, fraud detection systems, and transaction monitoring systems, among others. This can help banks comply with regulations, detect suspicious activities, and prevent fraud.</p>
<p>In the securities industry, data is crucial for investment decisions, risk management, and portfolio optimization. Canner Enterprise can help securities firms access and analyze data from various sources, such as market data providers, research firms, and trading systems. This can help securities firms make informed investment decisions and manage risks effectively.</p>
<h2 id="scenarios-of-using-data-virtualization-in-finance">Scenarios of using Data Virtualization in finance:</h2>
<ol>
<li><strong>Risk management</strong>: Banks and financial institutions can use data virtualization to access and integrate data from multiple sources such as credit bureaus, regulatory agencies, and internal systems to gain a more comprehensive view of their risk exposure. By virtualizing data, they can quickly and easily access data in real-time, perform simulations and stress tests, and make more informed decisions.</li>
<li><strong>Customer analytics</strong>: Financial institutions can use data virtualization to combine data from customer touchpoints such as customer service systems, transaction systems, and marketing databases to gain a more comprehensive view of their customers. By virtualizing data, they can quickly and easily analyze customer behavior, identify patterns, and improve customer engagement.</li>
<li><strong>Compliance reporting</strong>: Financial institutions can use data virtualization to access and integrate data from various systems and sources to ensure compliance with regulations. By virtualizing data, they can quickly generate compliance reports, monitor and audit transactions, and provide real-time access to regulatory data.</li>
<li><strong>Portfolio management</strong>: Investment firms can use data virtualization to access and integrate data from various sources, such as market data feeds, research reports, and internal systems to make more informed investment decisions. By virtualizing data, they can quickly analyze market trends, assess risks, and optimize their portfolios.</li>
<li><strong>Fraud detection</strong>: Financial institutions can use data virtualization to access and integrate data from various systems such as transaction, account opening, and fraud detection systems to detect and prevent fraudulent activity. By virtualizing data, they can quickly identify patterns, track suspicious activity, and prevent fraud before it occurs.</li>
</ol>


Canner Enterprise offers a data access platform that leverages data virtualization technology. In finance sector, it can be used for risk management, customer analytics, compliance reporting, portfolio management, and fraud detection.

Data Virtualization in Finance

<p>Flexible work arrangements are one of the main reasons why many talented people are attracted to startups. Distance is no longer a problem to consider when working remotely - as long as you have a computer and internet, you can work anytime and anywhere. Especially in the midst of the COVID-19 pandemic, remote work has become a new way of life.</p>
<p>Canner has always been committed to pursuing a work style that is most friendly and respectful to all team members. After a period of experimentation and trial, we have developed Canner&#39;s unique &quot;hybrid remote work model&quot; where team members come to the office twice a week.</p>
<h2 id="going-in-office-two-days-a-week">Going in office two days a week.</h2>
<p>During these office days, we have meetings where we discuss important information and brainstorm ideas face-to-face. We also have casual conversations over lunch or afternoon tea, which is a great way to build relationships and foster a positive work environment. Changing the work environment can also stimulate new ideas and creativity.</p>
<p><img src="/static/images/ws-post/2022_08_18_canner_life/canner_life_birthday.jpg" alt="canner life birthday"></p>
<p>On the other days, team members work remotely from home. This saves commuting time and provides flexibility in choosing a work location, whether it be in the city or in their hometown. We also use Gather, a virtual meeting space, to connect with remote team members and have discussions or brainstorming sessions.</p>
<p><img src="/static/images/ws-post/2022_08_18_canner_life/canner_use_gather.png" alt="canner use gather.png"></p>
<p>To ensure effective communication and team cohesion, Canner has established guidelines for remote work. These guidelines include Text Discuss, Video Discuss, Transparent Discussion, and Proactive Check. By following these guidelines, we can avoid communication difficulties and conflicts.</p>
<p>Remote culture is built on trust among team members. Even though we work remotely, our work quality is not affected. At Canner, we work in a casual but responsible manner, and we believe that our unique approach to remote work is a key factor in maintaining positive relationships within the team.</p>


I've heard that Canner employees only need to go into the office for two days a week, but their productivity doesn't suffer at all. That's amazing! You may wonder how we manage to achieve that. 

【Life@Canner】Canner Remote Culture

<p>The PostgreSQL wire protocol is a protocol used to communicate between client applications and the PostgreSQL database server. It defines a set of messages that are used to perform a variety of tasks, from establishing connections to executing SQL commands and fetching result sets. By using client libraries that implement the protocol, developers can easily connect to a PostgreSQL database and take advantage of its robust features and performance.</p>
<p>PostgreSQL wire protocol is important in the data ecosystem for several reasons:</p>
<ol>
<li><strong>Client-Server Communication</strong>: The PostgreSQL wire protocol defines the rules for communication between a PostgreSQL client and server. This is important because it allows applications and tools to connect to and interact with the PostgreSQL database, retrieve data, and perform database operations.</li>
<li><strong>Standardization</strong>: The PostgreSQL wire protocol is a standard, which means that it can be used by different tools and applications that support the protocol. This ensures that the data can be accessed and processed in a consistent and standardized way across different systems.</li>
<li><strong>Security</strong>: The PostgreSQL wire protocol provides a secure way to transmit data between the client and server by using SSL/TLS encryption. This ensures that sensitive data is protected during transmission, and makes it less vulnerable to interception or attacks.</li>
<li><strong>Performance</strong>: The PostgreSQL wire protocol is designed to be efficient, which means that it can transmit data quickly and with minimal overhead. This is important for applications that need to process large amounts of data quickly.</li>
<li><strong>Extensibility</strong>: The PostgreSQL wire protocol is extensible, which means that new features can be added to it as needed. This allows the protocol to evolve over time and support new use cases and requirements.</li>
</ol>
<h1 id="the-adoption-rate-of-postgresql-wire-protocol">The adoption rate of PostgreSQL wire protocol</h1>
<p>It&#39;s difficult to determine the exact adoption rate of PostgreSQL wire protocol because it is a widely used protocol and there is no central repository or authority that tracks its usage. However, we can make some general observations about its adoption based on various sources of information.</p>
<p>According to the DB-Engines ranking, PostgreSQL is the fourth most popular database management system as of February 2023.  Another indication of the adoption rate of the PostgreSQL wire protocol is the number of tools and applications that support it. Many popular tools and frameworks, such as JDBC, ODBC, provide support for the PostgreSQL wire protocol, making it accessible to a wide range of developers.</p>
<h1 id="databases-based-on-postgresql-wire-protocol">Databases based on PostgreSQL wire protocol</h1>
<p>There are several databases that are based on the PostgreSQL wire protocol. Some of the most popular ones include:</p>
<ol>
<li><strong>Amazon Redshift</strong>: Amazon Redshift is a data warehousing service based on PostgreSQL wire protocol. It is designed to handle large-scale data analytics workloads and provides a scalable and cost-effective solution for processing and analyzing large amounts of data.</li>
<li><strong>YugabyteDB</strong>: YugabyteDB is a distributed SQL database that supports PostgreSQL wire protocol. It is designed to provide high scalability, fault tolerance, and consistency, making it a popular choice for modern cloud-native applications.</li>
<li><strong>Citus</strong>: Citus is a distributed database that extends PostgreSQL with features such as horizontal scaling and sharding. It supports PostgreSQL wire protocol and is designed to handle large-scale, real-time workloads.</li>
<li><strong>CockroachDB</strong>: CockroachDB is a distributed SQL database that supports PostgreSQL wire protocol. It is designed to provide high availability, scalability, and performance, and is often used for modern cloud-native applications.</li>
<li><strong>TimescaleDB</strong>: TimescaleDB is a time-series database that extends PostgreSQL with features such as time-based partitioning and indexing. It supports PostgreSQL wire protocol and is designed to handle large-scale, time-series data workloads.</li>
</ol>
<p>These databases are based on PostgreSQL wire protocol, which means that they can be accessed and used using the same tools and applications that support PostgreSQL. This makes them a popular choice for organizations that are already familiar with PostgreSQL and want to extend its functionality or scalability.</p>
<h1 id="canner-enterprise-adopt-the-postgresql-wire-protocol">Canner Enterprise adopt the postgresql wire protocol</h1>
<p>Canner Enterprise leverages PostgreSQL wire protocol to bring the best connectivity to end applications, such as BI, AI, and data analytics software, Canner Enterprise allows you to create a virtual data access layer that sits between your application and data sources, enabling real-time access to data across multiple sources without the need for data movement or replication. </p>
<p>This approach can significantly reduce the complexity and cost of managing data integration and provides a more flexible and scalable way to access and analyze data. With Canner Data Virtualization, you can easily connect to a wide range of data sources, including cloud-based and on-premises databases, and create a unified view of data that can be easily accessed by your applications. Whether you&#39;re dealing with big data, real-time data processing, or complex data integration requirements, Canner Data Virtualization can help you streamline your data architecture and improve your data access and analysis capabilities.</p>


PostgreSQL wire protocol is a widely adopted standard protocol for communication between client applications and the PostgreSQL database server. Canner Enterprise adopted it to provide real-time access to data across multiple sources.

Why is the PostgreSQL wire protocol important?

<p>Today&#39;s data environment is becoming increasingly complex, and enterprises need to be able to quickly access and integrate data from various different formats and sources. To meet this demand, many data integration technologies have emerged, including Data Virtualization and Data Fabric. Although both can provide solutions for data integration, there are some key differences between them. This article will explore what sets Data Virtualization apart from Data Fabric.</p>
<h2 id="data-virtualization">Data Virtualization</h2>
<p>Data Virtualization is an enterprise integration technology that distributes data across different systems. It provides a unified data view through a virtual layer and can integrate data into multiple applications. Data Virtualization technology abstracts different data sources, allowing enterprises to easily access data without worrying about where it is stored. Using Data Virtualization technology can also increase data availability and integrity since data is integrated into a single view.</p>
<h2 id="data-fabric">Data Fabric</h2>
<p>Data Fabric is an enterprise integration technology that also distributes data across different systems, but it places more emphasis on the overall structure of data. Data Fabric technology allows enterprises to move and manage data freely between different data sources. It can provide a unified data management platform for enterprises, thereby increasing data availability and reliability. Using Data Fabric technology can also increase enterprise control over data and enhance data security.</p>
<h2 id="differences-between-data-virtualization-and-data-fabric">Differences between Data Virtualization and Data Fabric</h2>
<p>Data Virtualization is a transparent way of integrating distributed data. It typically integrates multiple data sources (such as relational databases, NoSQL databases, web services, and other data sources) into a virtual database, allowing users to access data from all data sources through a single interface. This provides a unified way of accessing data, reduces the need for data replication and synchronization, and improves the efficiency of data integration.</p>
<p>Data Fabric is a more comprehensive architectural model that aims to achieve an overall structure for enterprise data. Data Fabric usually refers to a unified data structure that can integrate different data sources and provide users with a unified data access interface. The goal of Data Fabric is to provide an overall data structure that makes it easier for enterprises to manage and control data, as well as to facilitate data analysis and application development.</p>
<p>In short, Data Virtualization focuses more on data integration technology, with a focus on achieving transparent integration of distributed data; while Data Fabric focuses more on data integration architecture, with a focus on achieving an overall structure for enterprise data that makes it easier for enterprises to manage and control data.</p>
<h2 id="implementing-data-fabric-with-canner-enterprise">Implementing Data Fabric with Canner Enterprise</h2>
<p>Data Virtualization technology and Data Fabric architectual concept are important innovation for modern enterprises in data integration. Canner Enterprise designs highly scalable and supportive Data Virtualization technology to assist enterprises in implementing Data Fabric architecture.</p>


This article explains the differences between Data Virtualization and Data Fabric. Data Virtualization focuses on distributing data through a virtual layer to provide a unified data view, while Data Fabric emphasizes overall data structure and offers a unified data management platform.

Data Virtualization vs. Data Fabric: What are the Differences?

<p>Data virtualization is a technology that integrates data from different sources into a virtual database. This technology helps businesses manage and analyze data better, increasing business efficiency and decision-making capabilities. In recent years, with the continuous expansion of data scale and the increase of data sources, the importance of data virtualization has become increasingly recognized and valued. This article will explore the importance of data virtualization and its future prospects.</p>
<h2 id="importance-of-data-virtualization-in-enterprises">Importance of Data Virtualization in Enterprises</h2>
<p>Data virtualization has many benefits. Firstly, it can help businesses better manage data, thus increasing business efficiency. By integrating data from different sources into a virtual database, businesses can easily access and analyze this data without the need for complex data transformation and integration work. This can save a lot of time and manpower, thereby increasing the efficiency and competitiveness of businesses.</p>
<p>Here are several key applications in enterprises:</p>
<ol>
<li>Simplify data integration: Different departments and systems within an enterprise may have unique data formats and structures, making data integration complex and time-consuming. Data virtualization can integrate and consolidate data without duplicating it, simplifying the data integration process.</li>
<li>Enhance data availability: Through data virtualization, businesses can centralize data from different sources into a virtual data layer, enhancing data availability and accessibility. This enables businesses to use and share data more easily, thereby increasing productivity and improving the quality of decision-making.</li>
<li>Improve data security: Data virtualization can help businesses keep data in the source system, reducing the risk of data leaks and security issues. Additionally, by setting permissions and controlling access, businesses can better protect their sensitive data.</li>
<li>Optimize data management: Data virtualization can help businesses better manage their data, improving data quality and reliability. Through data virtualization, businesses can organize and manage their data without disrupting existing data structures and systems.</li>
<li>Reduce costs and risks: Through data virtualization, businesses can reduce the cost of data integration and management. Additionally, as data virtualization does not require data to be copied and transferred between different systems, it can lower the risk of data loss and inconsistency.</li>
</ol>
<h2 id="future-of-data-virtualization">Future of Data Virtualization</h2>
<p>Data virtualization will have broader applications in the future. Firstly, as artificial intelligence and big data technologies develop, businesses will need more data for analysis and prediction. Data virtualization can help businesses better manage this data, thereby increasing their predictive and decision-making capabilities. For example, businesses can use data virtualization technology to integrate data from different sources into a virtual database and then use artificial intelligence technology to analyze and predict this data, resulting in more accurate results.</p>
<ol>
<li>Big data and AI: Data virtualization can help businesses better manage and integrate big data, improving the application of data analysis and AI technology. Through data virtualization, businesses can easily integrate data from multiple sources into a virtual data layer, creating larger datasets and achieving more accurate analysis and prediction.</li>
<li>Cloud data and multi-cloud architecture: As businesses and organizations increasingly adopt cloud data and multi-cloud architecture, data virtualization will become more important. Data virtualization can help businesses integrate data from different cloud platforms into a unified data layer, thereby improving data accessibility and availability.</li>
<li>Data privacy and compliance: As data privacy and compliance issues become more important, data virtualization can help businesses better protect sensitive data and ensure the legality of data use. By setting permissions and controlling access, businesses can better control data access and use, thereby ensuring data privacy and compliance.</li>
<li>IoT and edge computing: As the application of the Internet of Things (IoT) and edge computing technologies becomes more widespread, data virtualization will play an important role. Data virtualization can help businesses better manage and integrate data dispersed across different devices and edge computing nodes, achieving more effective data analysis and prediction.</li>
</ol>
<h2 id="canner-enterprise-data-virtualization-technology">Canner Enterprise Data Virtualization Technology</h2>
<p>Data virtualization in Canner Enterprise is an important new technology that can help businesses better manage and analyze data, improving predictive and decision-making capabilities. In the future, data virtualization will have broader applications, helping businesses address various challenges and opportunities. Therefore, businesses should strengthen their learning and application of data virtualization technology, thereby improving their competitiveness and innovation.</p>


Data virtualization can simplify data integration, enhance data availability, improve data security, optimize data management, and reduce costs and risks. In the future, data virtualization will have broader applications discuss in the article.

Importance and Future of Data Virtualization

<p>Most of the members in Canner come from the R&amp;D team. The process of developing products is both interesting and challenging, but there are times when we hit a wall and run out of ideas.</p>
<p>Effective team collaboration is one of the important keys to a company&#39;s success and continued growth. Through fun Team Building activities, team members can step out of their daily office environment, think outside the box, and not only improve communication among team members, but also enhance their ability to think from others&#39; perspectives and work together as a team.</p>
<p><img src="/static/images/ws-post/2022_11_03_canner_outing/pic.png" alt="Pic"></p>
<p>Besides, during the busy work schedule and the mask-wearing situation in the office, we have fewer opportunities to communicate with colleagues from different teams, which can make us feel like familiar strangers.</p>
<p>As the pandemic situation gradually improves, to promote interactions between different teams, we decided to have an Outing Team Building activity in the beautiful nature of Yilan. We planned a series of indoor and outdoor activities, allowing everyone to relax a bit from their busy work and get to know colleagues from different teams.</p>
<p><img src="/static/images/ws-post/2022_11_03_canner_outing/pic_1.png" alt="pic_1"></p>
<p>The first stop away from the office was in Dong&#39;ao, Yilan, where we tried the popular SUP (Stand-up paddleboarding) activity. Before getting into the water, everyone was excited!</p>
<p><img src="/static/images/ws-post/2022_11_03_canner_outing/pic_2.png" alt="pic_2"></p>
<p>It turned out that the SUP activity required teamwork and cooperation. Only when everyone worked together and coordinated with each other could we form a line of paddleboards. We also needed to take care of colleagues who were not good at water activities, so that they could have fun and feel safe.</p>
<p><img src="/static/images/ws-post/2022_11_03_canner_outing/pic_3.png" alt="pic_3"></p>
<p>In this Outing, we not only cultivated better team rapport but also rewarded everyone for their hard work over the year. Well done!</p>
<p><img src="/static/images/ws-post/2022_11_03_canner_outing/pic_4.png" alt="pic_4"></p>
<p>The brain-teasing team games in Team Building not only tested our IQ but also our teamwork skills XD</p>
<p><img src="/static/images/ws-post/2022_11_03_canner_outing/pic_5.png" alt="pic_5"></p>
<p>After a night&#39;s rest, we continued our adventure along the Nan&#39;ao coastline, riding ATVs to enjoy the scenery and clear our minds.</p>
<p>From not being able to know everyone&#39;s name within the first month of joining the company to immediately calling out the correct name after two days and one night of team building, that&#39;s the magic of actual interactions! Apart from consolidating team cohesion, we gained many benefits, such as:</p>
<h3 id="stepping-out-of-the-comfort-zone-and-discovering-different-aspects-of-ourselves-and-others">Stepping out of the comfort zone and discovering different aspects of ourselves and others</h3>
<p>Everyone has their own habits and preferences. Usually, during leisure time, we choose activities that we enjoy and are good at. However, in Team Building, we do activities that we may not normally engage in and talk to people from different backgrounds, which can inspire different ideas or find common topics.</p>
<h3 id="taking-a-break-and-coming-back-with-a-better-self">Taking a break and coming back with a better self</h3>
<p>As mentioned at the beginning of the article, we sometimes face obstacles that are difficult to overcome at work and get stuck in an infinite loop. Putting down the computer and taking a break, chatting aimlessly with team members, and looking at the boundless sea view can recharge us, and we can return to work with a better self and continue to strive!</p>
<h3 id="creating-a-group-memory">Creating a group memory</h3>
<p>A journey is made up of many exciting memories. A memorable activity adds a common memory to the entire team. During breaks, we occasionally talk about the muscle soreness caused by a large amount of outdoor activities, but we also remember the feeling of unity and laughter during the activities. With this memory that belongs only to us, our relationship has improved even more.</p>
<p><strong>In addition to static photos for memories, dynamic videos can also bring you an immersive experience</strong></p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/SoWbfkfHRtE" title="YouTube video player" frameborder="0"></iframe>

Teamwork makes the dream work! It's one of the most important keys to success and growth for any company. By planning fun Team Building activities, we can help our team members break away from their usual work environment.

【Life@Canner】Team Building： Think Outside The Box

<p>We are excited to announce the release of <strong><a href="https://vulcansql.com">VulcanSQL</a></strong>, an open-source Analytics API generator that enables data engineers to build scalable analytics APIs using only SQL without writing any backend code.</p>
<p>Analytics APIs are the primary interface for data consumers to utilize data in their daily business applications, such as BI, reports, dashboards, spreadsheets, and web applications. However, data stored in data warehouses are not accessible for those users and tools without an API consumption layer. VulcanSQL aims to solve that problem by translating SQL into flexible APIs. It is contextual in that it can translate APIs into the corresponding SQL based on different user personas and business contexts.</p>
<p><img src="/static/images/ws-post/2022_11_03_vulcansql_announce/vulcansql_diagram.png" alt="Vulcansql diagram"></p>
<p>With VulcanSQL, data engineers can quickly and easily create APIs without having to write any backend code. This makes it possible to accelerate development time and reduce the complexity of building data APIs. The generated APIs are highly scalable and performant, making it possible to serve thousands of requests per second without any degradation in performance.</p>
<p>VulcanSQL is designed to be highly customizable and extendable. It comes with built-in support for complex SQL queries and custom business logic, making it possible to create highly tailored APIs that meet specific business requirements. This flexibility makes it possible to create highly sophisticated data API layers that integrate seamlessly with existing business applications and data warehouses.</p>
<ul>
<li>Github: <a href="https://github.com/Canner/vulcan-sql">https://github.com/Canner/vulcan-sql</a></li>
<li>Website: <a href="https://vulcansql.com">https://vulcansql.com</a></li>
<li>Discord: <a href="https://discord.gg/8JAsVFfNes">https://discord.gg/8JAsVFfNes</a></li>
</ul>


We are excited to announce the release of VulcanSQL, an open-source Analytics API generator that enables data engineers to build scalable analytics APIs using only SQL without writing any backend code.

VulcanSQL: Open-source Analytics API Generator

<p>Modern data stack is a technology stack used to collect, process, store, and analyze data, usually composed of multiple tools and platforms. Its purpose is to help businesses process data more effectively and improve the speed and quality of data analysis.</p>
<p>The popularity of modern data stack is related to the modern enterprise&#39;s need to process large amounts of data from multiple sources, including websites, mobile applications, social media, sensors, servers, and other databases. Furthermore, businesses need to transform this data into useful insights to make better business decisions.</p>
<p>Traditional data stacks typically require the use of multiple different tools and technologies to complete various stages of data processing. These tools are often incompatible with each other and require a lot of manual configuration and maintenance. On the other hand, modern data stacks are more automated and scalable, usually composed of tools and platforms that can be integrated and upgraded more easily, and generate useful data analysis insights more quickly.</p>
<p>Therefore, modern data stack has become increasingly popular in recent years, especially in large and data-driven enterprises that need to process large amounts of data to improve their business efficiency and competitiveness.</p>
<h2 id="what-is-modern-data-stack">What is Modern Data Stack?</h2>
<p>Modern Data Stack refers to a set of modern technologies, tools, and processes that work together to help businesses collect, store, process, and analyze data in real-time, enabling data-driven decision-making.</p>
<p>The modern data stack includes several components, such as:</p>
<ol>
<li>Data sources: These are the systems or platforms where data is generated or collected, such as databases, applications, APIs, and IoT devices.</li>
<li>Data pipelines: These are the tools and processes used to extract data from the data sources, transform it into a usable format, and load it into a data warehouse or data lake. Examples of data pipeline tools include Apache Kafka, Apache NiFi, and Fivetran.</li>
<li>Data warehouse or data lake: These are the storage systems where data is stored and made accessible for analysis. Examples of data warehouse systems include Amazon Redshift, Google BigQuery, and Snowflake. Examples of data lake systems include Apache Hadoop and Amazon S3.</li>
<li>Business intelligence and analytics tools: These are the tools used to explore and analyze data, create dashboards and reports, and gain insights from data. Examples of BI and analytics tools include Tableau, Looker, and Power BI.</li>
<li>Data governance and security tools: These are the tools used to ensure that data is properly secured, comply with regulations, and manage access to data.</li>
</ol>
<h2 id="why-is-modern-data-stack-so-popular">Why is Modern Data Stack so popular?</h2>
<p>With the explosive growth of data, enterprises need more data analysis and data science support to help make better decisions. Traditional data analysis processes often require a lot of time and manpower, especially in data extraction and transformation, which often requires technical assistance. The emergence of the Modern Data Stack allows non-technical personnel to easily perform data analysis, thereby improving the efficiency of data analysis and saving costs.</p>
<p>In addition, Modern Data Stack has the following advantages:</p>
<ol>
<li>Scalability: Modern data stack technologies are designed to scale seamlessly as data volume and complexity grow, enabling businesses to process and analyze vast amounts of data quickly and easily.</li>
<li>Real-time data processing: The modern data stack enables businesses to process and analyze data in real-time, providing up-to-the-minute insights and enabling faster decision-making.</li>
<li>Integration: The modern data stack is designed to be highly integrated, enabling businesses to easily connect data sources, data pipelines, and analytics tools.</li>
<li>Democratization of data: The modern data stack makes it easier for businesses to share data and insights across teams and departments, enabling a more data-driven culture and empowering individuals to make data-driven decisions.</li>
<li>Flexibility: The modern data stack is designed to be flexible, enabling businesses to easily add or remove components as their needs change and new technologies emerge.</li>
</ol>
<h2 id="why-should-enterprises-pay-attention-to-modern-data-stack">Why should enterprises pay attention to Modern Data Stack?</h2>
<ol>
<li>Competitive advantage: The modern data stack enables businesses to extract valuable insights from their data, providing a competitive advantage in a rapidly changing business landscape.</li>
<li>Improved decision-making: By leveraging the modern data stack, enterprises can make data-driven decisions quickly and effectively, leading to better business outcomes.</li>
<li>Increased efficiency: The modern data stack automates many of the data processing and analysis tasks, enabling businesses to be more efficient and productive.</li>
<li>Better customer experiences: The modern data stack enables businesses to better understand their customers, leading to more personalized and targeted experiences that drive loyalty and retention.</li>
<li>Compliance and governance: The modern data stack includes tools for compliance and governance, ensuring that enterprises remain compliant with regulations and protect the privacy of their customers.</li>
<li>Innovation: The modern data stack enables enterprises to experiment with new data sources, analysis techniques, and technologies, leading to new insights and opportunities for innovation.</li>
</ol>
<h2 id="using-canner-as-a-data-access-layer-of-the-modern-data-stack">Using Canner as a Data Access Layer of the Modern Data Stack</h2>
<p>Enterprises can benefit from data access layers in many ways, including:</p>
<ol>
<li>Improved data quality and consistency: The data access layer can help ensure data consistency and accuracy in different tools and applications, reducing the risk of errors and inconsistencies in data analysis.</li>
<li>Increased production efficiency: The data access layer provides a unified interface for accessing and analyzing data, reducing the need for redundant data processing and improving the efficiency of data analysis. This can increase productivity and reduce costs.</li>
<li>Enhanced security: The data access layer can help ensure secure and controlled use of data, minimizing the risk of data leakage and unauthorized access to sensitive data.</li>
<li>Scalability: The data access layer is designed to be highly scalable, able to handle large amounts of data and support multiple concurrent users. This can help enterprises expand their data capabilities over time as needed.</li>
<li>Flexibility: The data access layer can be customized according to the specific needs and requirements of the enterprise, allowing the organization to use its data in ways that suit its unique needs and goals.</li>
<li>Better decision-making: By providing a unified interface for accessing and analyzing data, the data access layer can help enterprises generate insights more quickly and make more informed, data-driven decisions.</li>
</ol>


Modern data stack helps businesses process data more effectively and improve the speed and quality of data analysis. The blog discusses the components of modern data stack and its benefits to enterprises.

The Modern Data Stack: A Comprehensive Overview and Benefits for Enterprises

<p>Moving data from an on-premise environment to the cloud can be a complex process with several challenges that need to be addressed. Here are some key challenges to consider:</p>
<ol>
<li><strong>Data security</strong>: One of the main challenges of moving data to the cloud is ensuring data security. The data must be protected from unauthorized access during transit and at rest in the cloud. In addition, there must be measures in place to ensure data privacy and compliance with regulations.</li>
<li><strong>Data integration</strong>: Data integration can be challenging when moving data from an on-premise environment to the cloud. The data may be stored in different formats, and it may be difficult to integrate the data with other systems in the cloud.</li>
<li><strong>Data latency</strong>: Data latency can be an issue when moving data to the cloud. The distance between the on-premise environment and the cloud can cause delays in data transmission, which can impact application performance.</li>
<li><strong>Data governance</strong>: When moving data to the cloud, it is important to maintain policies to ensure data is used appropriately. This can include access controls, auditing, and compliance monitoring.</li>
</ol>
<h2 id="how-data-fabric-addresses-challenges-in-moving-data-to-the-cloud"><strong>How Data Fabric Addresses Challenges in Moving Data to the Cloud</strong></h2>
<p>Data fabric can help address these challenges by providing a unified, virtualized view of the data that spans both on-premise and cloud environments. Data fabric enables data to be accessed and integrated from multiple sources, regardless of where the data is stored. This can help address data integration challenges by providing a single view of the data, regardless of where it is stored. In addition, data fabric can help address data latency challenges by caching frequently accessed data in memory, reducing the need to access the data across the network.</p>
<p>Data fabric also provides a layer of abstraction that can help address data security and governance challenges. The data can be protected and managed in a consistent way, regardless of where it is stored. This can help ensure data security and compliance with regulations.</p>
<h1 id="real-world-examples">Real-world examples</h1>
<p>here are some real-world examples of how data fabric is being used to address the challenges of moving data from on-premise environments to the cloud:</p>
<ol>
<li><strong>Financial services</strong>: A global bank wanted to move their data from an on-premise environment to the cloud to reduce costs and improve scalability. However, they faced challenges with data integration and security. By using a data fabric platform, the bank was able to create a unified view of their data that spanned both environments, enabling them to easily move their data to the cloud while maintaining data security and compliance.</li>
<li><strong>Healthcare</strong>: A healthcare provider wanted to move their data to the cloud to improve collaboration and analytics. However, they faced challenges with data latency and governance. By using a data fabric platform, the provider was able to create a unified view of their data that spanned both on-premise and cloud environments, enabling them to access their data in real-time and maintain data governance policies across both environments.</li>
<li><strong>Retail</strong>: A large retailer wanted to move their data to the cloud to improve customer insights and personalization. However, they faced challenges with data integration and latency. By using a data fabric platform, the retailer was able to create a unified view of their data that spanned both environments, enabling them to easily integrate their data and access it in real-time, improving their customer insights and personalization capabilities.</li>
<li><strong>Manufacturing</strong>: A manufacturer wanted to move their data to the cloud to improve supply chain management and analytics. However, they faced challenges with data integration and security. By using a data fabric platform, the manufacturer was able to create a unified view of their data that spanned both environments, enabling them to easily integrate their data and maintain data security and compliance policies.</li>
</ol>


This post discusses challenges of moving data to the cloud and how data fabric can address them, we also provide real-world examples.

Moving Data to the Cloud with Data Fabric in industries

<p>Canner, the data access company, is excited to announce the release of their new documentation support in both English and Traditional Chinese. This new addition is a significant step towards making Canner&#39;s platform more accessible and user-friendly for people all around the world.</p>
<p>Canner is a powerful data access platform that enables businesses and organizations to collect, manage, and analyze data efficiently. With this new documentation support in both English and Traditional Chinese, users will have the opportunity to learn more about Canner and how to use its powerful features and functionalities in their preferred language.</p>
<img src="/static/images/ws-post/2023_01_03_docs_en_zh/preface.png"/>

<p>&quot;We understand the importance of providing our users with comprehensive support in their preferred language,&quot; said the Canner team. &quot;Our goal is to make Canner&#39;s platform accessible to users all around the world, and this new documentation support is a major step towards achieving that goal.&quot;</p>
<p>The new documentation support provides users with a comprehensive guide to using Canner&#39;s platform. It covers all aspects of the platform, including how to get started, how to set up data sources, how to manage data, and how to analyze data using Canner&#39;s powerful tools and features. The documentation also includes helpful tips and best practices for using Canner&#39;s platform effectively.</p>
<p>To learn more about Canner and its new documentation support in English and Traditional Chinese, please visit <a href="https://docs.cannerdata.com">https://docs.cannerdata.com</a>. The Canner team is always available to provide support and answer any questions that users may have about the platform.</p>


Canner, the data access company, has announced the release of new documentation support in both English and Traditional Chinese. This new addition is a major step towards making Canner's platform more accessible and user-friendly for people all around the world.

Canner Launches New Docs Support

<p>DBLink and data virtualization are two different technologies that are used to access and integrate data from multiple sources, but they have some key differences.</p>
<p>DBLink is a feature in many relational database management systems (RDBMS) that allows tables in one database to be accessed from another database. DBLink works by establishing a connection between two databases, enabling SQL statements to be executed across the two databases. DBLink is primarily used for accessing data within RDBMS systems, and it requires both databases to be running on the same network.</p>
<p>Data virtualization, on the other hand, is a technology that allows data to be accessed and integrated from multiple sources, regardless of where the data is located. Data virtualization provides a virtualized view of the data, which enables users to access and analyze the data without physically moving or replicating the data. Data virtualization works by creating a virtual layer that integrates data from multiple sources and presents it as a single view, which users or applications can access.</p>
<p>The key difference between DBLink and data virtualization is that DBLink is limited to accessing data within RDBMS systems, while data virtualization can be used to integrate data from any source, including RDBMS systems, NoSQL databases, cloud storage systems, and APIs. Data virtualization also provides a more flexible and scalable solution for data integration, as it does not require data to be moved or replicated, and it can be used to access and integrate data in real-time.</p>
<h1 id="what-are-the-problems-of-using-dblink">What are the problems of using DBLink</h1>
<p>While DBLink can be a useful feature for accessing data across multiple databases within a relational database management system (RDBMS), there are several challenges and limitations to using DBLink. Here are some of the problems of using DBLink:</p>
<ol>
<li><strong>Performance</strong>: DBLink can impact system performance when accessing data across different databases, as it involves additional network traffic and processing overhead. This can lead to slower query response times and impact the performance of the database system.</li>
<li><strong>Security</strong>: When using DBLink, users need to have the appropriate permissions to access the data in the remote database. This can raise security concerns, as it involves granting users access to data in another database.</li>
<li><strong>Compatibility</strong>: DBLink may not be compatible with all RDBMS systems, which can limit its usefulness for organizations using multiple databases.</li>
<li><strong>Maintenance</strong>: DBLink requires ongoing maintenance and management, including monitoring for errors and issues, and updating configurations as needed.</li>
<li><strong>Scalability</strong>: DBLink may not be scalable for larger data sets or for organizations with complex data integration requirements. As the number of databases and tables increases, managing DBLink connections can become more complex.</li>
</ol>
<p>Organizations need to carefully consider the performance, security, compatibility, maintenance, and scalability implications of using DBLink before implementing it. Alternative solutions, such as data virtualization, may offer a more flexible and scalable approach for data integration that addresses some of the limitations of DBLink.</p>
<h1 id="can-data-virtualization-help">Can data virtualization help?</h1>
<p>Data virtualization can be a useful solution for addressing some of the problems associated with using DBLink. Here&#39;s how:</p>
<ol>
<li><strong>Performance</strong>: Data virtualization can improve query performance by caching frequently accessed data in memory, reducing the need for data to be accessed across the network. This can improve query response times and reduce the impact on the performance of the database system.</li>
<li><strong>Security</strong>: Data virtualization provides a layer of abstraction that can help address security concerns by managing and controlling data access in a consistent way, regardless of where the data is located. This can help ensure that data is accessed only by authorized users, and that data privacy and compliance requirements are met.</li>
<li><strong>Compatibility</strong>: Data virtualization is designed to be compatible with a wide range of data sources, including relational and non-relational databases, cloud storage systems, and APIs. This provides a more flexible and scalable solution for data integration than DBLink.</li>
<li><strong>Maintenance</strong>: Data virtualization can help reduce maintenance requirements by providing a centralized view of data, eliminating the need to manage and maintain multiple connections to different databases. This can help reduce the complexity and cost of managing data integration.</li>
<li><strong>Scalability</strong>: Data virtualization can help improve scalability by providing a more flexible and scalable solution for data integration. As the number of data sources and tables increase, data virtualization can help manage the complexity of data integration and provide a more scalable solution.</li>
</ol>
<h2 id="with-canner-enterprise-the-semantic-layer">With Canner Enterprise, the Semantic Layer</h2>
<p>Canner Enterprise’s Semantic Layer can help address the challenges associated with using DBLink, providing a more flexible and scalable solution for data integration that can help improve query performance, data security, compatibility, maintenance, and scalability. By using Caner Enterprise, organizations can create a unified, virtualized view and metrics of their data that spans multiple sources, enabling them to make more informed decisions based on a more comprehensive view of their data.</p>


This post compares DBLink and data virtualization for accessing and integrating data from multiple sources, and explains how data virtualization can provide a more flexible and scalable solution for data integration.

DBLink vs. Data Virtualization: What are the Differences?

<p>In a data mesh architecture, data access is a critical component that enables data consumers to discover and access the data they need in a self-serve manner. However, this does not mean that data access is unregulated or uncontrolled. A data mesh architecture requires a different kind of more distributed and decentralized control.</p>
<p>In a traditional centralized data architecture, data access is typically controlled by a centralized data team, which can create bottlenecks in the data pipeline and slow down data access. This can be particularly problematic in large organizations with many different data consumers with different needs and requirements.</p>
<p>In contrast, in a data mesh architecture, data access is decentralized and self-serve, which allows data consumers to access the data they need directly. This is achieved through data discovery, data catalogs, and data access APIs. With these capabilities, data consumers can search for and discover relevant data sets, understand their structure and quality, and access them using a standardized interface.</p>
<h2 id="current-pain-points">Current pain points</h2>
<p>Without a data access layer in a data mesh architecture, there can be several pain points, including:</p>
<ol>
<li><strong>Data Inconsistency</strong>: Each data product team may use its own data storage, which can lead to inconsistent data across the organization. Without a centralized data access layer, it can be challenging to ensure that the data is up-to-date, accurate, and consistent.</li>
<li><strong>Data Security</strong>: Data security is a critical concern in any organization. Without a data access layer, there may be limited control over who can access data, how data is accessed, and how it is protected. This can increase the risk of data breaches, leaks, and unauthorized access.</li>
<li><strong>Data Governance</strong>: Data governance is essential for ensuring that data is properly managed, tracked, and audited. Without a centralized data access layer, it can be challenging to enforce data governance policies consistently across the organization.</li>
<li><strong>Data Integration</strong>: Data mesh architecture emphasizes the need for data interoperability between different data products. Without a data access layer, integrating data from different sources can be a complex and time-consuming process.</li>
<li><strong>Scalability</strong>: As the organization grows and the number of data products increases, managing and scaling without a centralized data access layer can become challenging. It can be difficult to ensure that all data products are performing optimally and meeting the organization&#39;s needs.</li>
<li><strong>Complexity</strong>: Without a centralized data access layer, the complexity of managing data products and maintaining consistency can increase. This can result in higher costs, longer development cycles, and slower time-to-market.</li>
</ol>
<h2 id="canner--the-data-access-layer-of-data-mesh"><strong>Canner — The Data Access Layer of Data Mesh</strong></h2>
<p>Canner’s &quot;Data Access Layer&quot; is designed around four core principles:</p>
<h3 id="1-data-virtualization"><strong>1. Data Virtualization</strong></h3>
<p>The Data Access Layer is collaborative and distributed, with each silo or data source independently scalable or aggregated together.</p>
<h3 id="2-data-productization"><strong>2. Data Productization</strong></h3>
<p>Transform data models into domain-oriented datasets, which can be owned by data owners, shared, and governed by open APIs. This allows for interchangeable metadata and access rules, letting data speak the language of your business.</p>
<h3 id="3-data-authorization"><strong>3. Data Authorization</strong></h3>
<p>Implement a consistent data authorization framework from data sources to data applications that is integrated with existing Identity and Access Management (IAM). This ensures that data authorization is consistent across data sources, IAM, and data applications.</p>
<h3 id="4-data-consumption"><strong>4. Data Consumption</strong></h3>
<p>Data consumers can generate queries and APIs with intent and contextual settings. These are applied to the corresponding datasets via intent declaration, and then delivered to target consumers for final analytics and display.</p>
<p>By enabling self-serve data access, a data mesh architecture can improve the speed and efficiency of data consumption while also promoting data democratization and collaboration. This means that different teams and individuals across the organization can access and use data in a more agile and flexible way, without relying on a centralized data team to provision and manage data access. Furthermore, allowing data consumers to access data directly can promote a culture of data-driven decision-making, where individuals and teams are empowered to make decisions based on data rather than gut instincts or incomplete information.</p>


This post discusses the importance of data access in a data mesh architecture, and how Canner's "Data Access Layer" addresses common pain points.

Why is data access important in data mesh?

<p>Canner Enterprise v2 focuses on providing a single platform for all data access processes, allowing for improved collaboration and interaction with data products within organizations. The platform features a redesigned data exploration process, data sharing process, and modular design tailored to different user personas.</p>
<p>&quot;Our vision at Canner is to make all data access processes possible on a single platform,&quot; said the company&#39;s CEO Howard. &quot;With Canner Enterprise v2, we are focused on improving the experience for data professionals and application users, enabling them to collaborate and interact with data products seamlessly.&quot;</p>
<p>Key features of Canner Enterprise v2 include a unified side panel for easy data exploration and access, as well as modular design that allows for different user personas to access different functionality based on their needs.</p>
<p>&quot;We are thrilled to launch Canner Enterprise v2 and provide a more streamlined and user-friendly platform for data professionals and application users,&quot; said the CEO. &quot;We believe that this new product will help organizations to better manage their data and collaborate more effectively, ultimately leading to improved decision-making and business outcomes.&quot;</p>
<p>For more information about Canner Enterprise v2, visit the company&#39;s website at <a href="https://cannerdata.com">https://cannerdata.com</a>.</p>


Today we announced the launch of Canner Enterprise v2, which builds on the success of the v1 product to provide an enhanced user experience for data professionals and application users.

Canner Enterprise v2 Officially Release

<p>Data mesh and data fabric are architectural approaches for managing data in large organizations, but they differ in several ways.</p>
<p>Data mesh is an architectural approach that emphasizes decentralization of data ownership and management, where each domain or product team is responsible for managing its own data. Data mesh also emphasizes the need for data interoperability and standardization between different domains, achieved through data contracts and APIs.</p>
<p>On the other hand, data fabric is an architectural approach that aims to provide a unified view of data across the organization, regardless of where the data is stored or managed. A data fabric typically uses a data virtualization layer to provide a single access point to all data sources, making integrating data across different systems easier.</p>
<h2 id="differences-between-the-two-architectures">Differences between the two architectures</h2>
<ol>
<li>Ownership and management of data:
In a data mesh architecture, each domain or product team manages its own data. For example, a team responsible for customer data would manage its own customer database, and a team responsible for product data would manage its own product database. On the other hand, in a data fabric architecture, data is centrally managed and governed, and the data fabric provides a unified view of all data across the organization, regardless of where the data is stored.</li>
<li>Data interoperability and standardization:
In a data mesh architecture, data interoperability and standardization are achieved through the use of data contracts and APIs, which define how data should be exchanged and accessed between different domains. Each domain or product team is responsible for defining its own data contracts and APIs. In contrast, a data fabric architecture typically uses a data virtualization layer to provide a unified view of all data across the organization, making integrating data from different sources easier.</li>
<li>Data governance and security:
In a data mesh architecture, data governance and security are decentralized, with each domain or product team responsible for managing its own data security and governance policies. Data governance and security are centrally managed and enforced across the organization in a data fabric architecture.</li>
<li>Scalability and flexibility:
Data mesh architecture is designed to be flexible and scalable, making it easier to add new domains or product teams as the organization grows. Each domain or product team can manage its own data infrastructure, allowing them to choose the tools and technologies that best fit its needs. In contrast, a data fabric architecture is designed to provide a unified view of all data across the organization, making it easier to integrate and analyze data. However, this can be less flexible and more challenging to scale as the organization grows.</li>
</ol>
<h2 id="implementing--with-canner-enterprise">Implementing  with Canner Enterprise</h2>
<p>Both Data Mesh and Data Fabric are important architectural concepts for modern enterprises in data integration. Canner Enterprise designs to assist enterprises in implementing both Data Mesh and Data Fabric architecture.</p>


This article explains the differences between Data Mesh and Data Fabric, two data integration technologies. Data Mesh emphasizes decentralization of data ownership and management, while Data Fabric aims to provide a unified view of all data across the organization.

Data Mesh vs. Data Fabric: What are the Differences?

<p>Headless BI (Business Intelligence) is an approach to delivering analytics and reporting capabilities without a user interface. In a headless BI system, the data and analytics are decoupled from the visualization layer, which means that the data can be accessed programmatically by other applications or services.</p>
<p>Headless BI allows developers to embed analytics functionality into their own applications or workflows, without requiring users to switch to a separate BI tool or platform. This approach enables developers to build customized data-driven applications or integrate analytics capabilities into existing workflows, which can improve the user experience and productivity.</p>
<p>In a headless BI system, the data is typically stored in a separate data warehouse or data lake, and the analytics are performed using specialized tools such as SQL or Python. The results of the analytics can then be accessed programmatically using APIs, which can be integrated into other applications or services.</p>
<p>Headless BI can be particularly useful in modern, cloud-based environments where data and analytics are distributed across multiple systems and services. It can help organizations to build flexible, scalable analytics solutions that can be customized to meet their specific needs. However, it may require more technical expertise than traditional BI systems, as developers need to understand how to access and manipulate data programmatically.</p>
<p>There are several benefits to using a headless BI approach:</p>
<ol>
<li><strong>Flexibility</strong>: With headless BI, developers can build custom analytics solutions that are tailored to their specific needs. They can choose the data sources, analytics tools, and APIs that best meet their requirements, and build the solution using the programming language and framework of their choice.</li>
<li><strong>Scalability</strong>: Headless BI systems can be scaled up or down as needed, depending on the volume of data and the number of users. Developers can add or remove data sources, analytics tools, and APIs as the needs of the organization change.</li>
<li><strong>Integration</strong>: Headless BI makes it easy to integrate analytics functionality into other applications and workflows. Developers can use APIs to access the analytics results programmatically and incorporate them into other systems or services.</li>
<li><strong>Cost-effective</strong>: Headless BI can be a more cost-effective solution than traditional BI tools, especially for organizations with complex or unique data requirements. Instead of paying for a full suite of BI tools, organizations can build custom solutions that meet their specific needs.</li>
<li><strong>Governance</strong>: With headless BI, organizations can maintain centralized control over their data, even as it is used in different applications and services. This can help ensure that data is consistent, accurate, and up-to-date, and can also help with compliance and regulatory requirements.</li>
</ol>
<h2 id="key-features-of-headless-bi">Key features of headless BI:</h2>
<ol>
<li><strong>Decoupled architecture</strong>: In a headless BI system, the data analytics and visualization layers are decoupled. This means that the data and analytics can be accessed programmatically, without the need for a dedicated user interface.</li>
<li><strong>API-driven</strong>: Headless BI relies on APIs (Application Programming Interfaces) to make the data and analytics available to other applications and services. APIs can be used to access data, perform analytics, and retrieve results.</li>
<li><strong>Customizable</strong>: Headless BI systems are highly customizable. Developers can choose the analytics tools and data sources that best meet their needs and build custom solutions that are tailored to their specific requirements.</li>
<li><strong>Developer-focused</strong>: Headless BI is focused on providing analytics capabilities to developers, rather than end-users. This means that it may require more technical expertise to implement and use, but it also provides greater flexibility and control over the analytics process.</li>
<li><strong>Governance</strong>: Headless BI systems provide centralized control over data, even as it is used in different applications and services. This can help ensure that data is accurate, consistent, and up-to-date, and can also help with compliance and regulatory requirements.</li>
</ol>
<h2 id="canner-enterprise-to-achieve-headless-bi-architecture">Canner Enterprise to achieve Headless BI Architecture</h2>
<p>By using Canner Enterprise, companies can achieve a true Headless BI architecture. This means that the BI layer is separated from the data layer, allowing companies to easily connect to any data source, regardless of where the data resides. This enables companies to achieve a more agile and flexible approach to BI, enabling them to quickly and easily adapt to changing business needs.</p>
<p>With Canner Enterprise, companies can easily connect to any data source, including both traditional and modern sources such as MS SQL, Databricks, SAP, Oracle, and Big Data platforms. Canner Enterprise&#39;s powerful data virtualization technology allows companies to combine and transform data from multiple sources, enabling them to create a single, unified view of their data.</p>
<p>Canner Enterprise also offers robust security features, allowing companies to authorize different groups and users to access specific data sources and BI content. This ensures that data is protected and that only authorized users have access to sensitive data.</p>
<p>Additionally, Canner Enterprise offers powerful optimization features that allow companies to optimize their BI architecture based on different use cases and scenarios. This enables companies to achieve advanced analytics and custom BI, allowing them to gain deeper insights into their data and make more informed business decisions.</p>


Headless BI is an innovative approach to delivering analytics and reporting capabilities without a user interface. It enables developers to embed analytics functionality into their own applications or workflows, improving the user experience and productivity.

What is headless BI?

<p>Lambda and Kappa Architecture are two popular approaches to designing large-scale data processing systems that can handle high volume, velocity, and variety of data. Here are the key differences between them:</p>
<h3 id="1-batch-vs-real-time-processing">1. Batch vs. Real-time Processing:</h3>
<p>Lambda Architecture involves processing data in both batch and real-time mode. In contrast, Kappa Architecture only processes data in real time.</p>
<h3 id="2-complexity">2. Complexity:</h3>
<p>Lambda Architecture is more complex than Kappa Architecture as it involves building and maintaining two separate processing pipelines: real-time data processing and batch processing. In contrast, Kappa Architecture has only one processing pipeline.</p>
<h3 id="3-data-storage">3. Data Storage:</h3>
<p>In Lambda Architecture, data is stored in two different storage systems: one for real-time processing and another for batch processing. Kappa Architecture, on the other hand, only requires one storage system, which can store both real-time and batch data.</p>
<h3 id="4-fault-tolerance">4. Fault Tolerance:</h3>
<p>Lambda Architecture is more fault-tolerant than Kappa Architecture because it has redundant processing pipelines, one for real-time data processing and another for batch processing. In case of any failure, the system can switch to the other pipeline. In contrast, Kappa Architecture has only one processing pipeline, which can result in data loss in case of any failure.</p>
<h3 id="5-scalability">5. Scalability:</h3>
<p>Both architectures are highly scalable and can handle large volumes of data. However, Lambda Architecture is better suited for scenarios where there is a high volume of data and the processing requirements are complex.</p>
<h2 id="which-is-better">Which is better?</h2>
<p>Lambda Architecture and Kappa Architecture are both useful for different scenarios and have their own advantages and disadvantages. The choice between the two architectures depends on the specific needs of the system being designed. Here are some factors that can help determine which architecture is better:</p>
<ol>
<li><strong>Real-time Processing</strong>: If the system requires real-time processing of data and the data processing requirements are not too complex, Kappa Architecture can be a better choice as it has a simpler design and allows for faster processing of data.</li>
<li><strong>Complex Data Processing</strong>: If the system requires complex data processing, involving both batch and real-time processing, Lambda Architecture can be a better choice as it allows for data processing in both modes and can handle complex data processing requirements.</li>
<li><strong>Fault Tolerance</strong>: If the system requires high fault tolerance and redundancy, Lambda Architecture can be a better choice as it has redundant processing pipelines, which can help in case of any failures.</li>
<li><strong>Data Storage</strong>: If the system requires efficient and cost-effective data storage, Kappa Architecture can be a better choice as it only requires one storage system for both real-time and batch data.</li>
<li><strong>Flexibility</strong>: If the system requires flexibility to handle different types of data sources and processing requirements, Lambda Architecture can be a better choice as it can handle both structured and unstructured data and provides the flexibility to handle complex processing requirements.</li>
</ol>
<h2 id="achieve-lambda-architecture-with-canner">Achieve Lambda Architecture with Canner</h2>
<p>Fortunately, Canner Enterprise provides a software solution that makes it easy to achieve Lambda Architecture. With Canner Enterprise, you can quickly build and deploy Lambda Architecture-based data processing systems, enabling you to process large volumes of data in a fault-tolerant and scalable way. Whether you&#39;re dealing with big data, real-time data processing, or complex data processing requirements, Canner Enterprise can help you achieve Lambda Architecture without the hassle of building and maintaining the system from scratch.</p>


This post compares Lambda and Kappa Architecture, two approaches to designing large-scale data processing systems.

Lambda vs. Kappa Architecture

<p>Many companies will hold a Kick-off meeting at the beginning of the new year. The Kick-off meeting is an important occasion to show the company&#39;s strength and declare its goals. Through this activity, we can strengthen teamwork, cultivate company culture, and continue to pass on these values. At Canner, we take this event very seriously as it is the beginning of a new chapter for us together!</p>
<p>As we begin this dynamic year, we are preparing for new challenges and opportunities. At the Kick-off Meeting, we reflect on the past year&#39;s accomplishments, discuss future plans and strategies, and strengthen teamwork and culture. This year&#39;s theme is &quot;I am a Canner, I am Possible.&quot; This theme expresses our confidence in ourselves, our team and our company. When we combine our ability, belief and determination, we can create the impossible!</p>
<h2 id="in-this-years-kick-off-meeting-we-are-divided-into-two-parts">In this year&#39;s Kick-Off Meeting we are divided into two parts:</h2>
<p>In the first half of the activity, each department shared the achievements of the past year and the goals and plans of this year with the colleagues of the whole company. It not only allows everyone to understand the operation and development of each team from the entire company level, Sales Team, Marketing Team, Dev Team, Support Team and Operation Team, but also allows company colleagues to better understand the cooperation and contributions between various departments.</p>
<p><img src="/static/images/ws-post/2023_04_02_2023_kickoff/IMG_1851.jpg" alt="IMG_1851.JPG"></p>
<p>In the second half of the event, we also invite colleagues from the R&amp;D team to share some amazing R&amp;D technologies and how they respond to market needs. These technologies and thought processes not only have an important impact on product development, but also reflect our professionalism and innovation in the field of technology. Through such exchanges and sharing, colleagues throughout the company have a better understanding of our products and feel our enthusiasm and commitment to products and technologies.</p>
<p><img src="/static/images/ws-post/2023_04_02_2023_kickoff/IMG_8145.jpg" alt="IMG_8z145.JPG"></p>
<p><img src="/static/images/ws-post/2023_04_02_2023_kickoff/IMG_4802.jpg" alt="IMG_4802.JPG"></p>
<p><img src="/static/images/ws-post/2023_04_02_2023_kickoff/IMG_3044_jpg_2.jpg" alt="IMG_3044_jpg 2.JPG"></p>
<p>This time we chose Taoyuan as the event location. This is not only to allow team members to leave the office environment, but also to inspire everyone&#39;s creativity in this way, and encourage everyone to adopt the &quot;think outside of the box&quot; way of thinking , concentrate on sharing and communicating in the kick-off meeting. In such an environment, every colleague can feel a new atmosphere and get more inspiration from it. In addition, we also set an interesting dress code theme on the day of the event: &quot;Canner&quot;. The activity is carried out in the form of a team competition. Each team member can stimulate each other&#39;s creativity, observation and association through team discussion, and can also promote the interaction and understanding between team members.</p>
<p><img src="/static/images/ws-post/2023_04_02_2023_kickoff/IMG_1855.jpg" alt="IMG_1855.JPG"></p>
<h3 id="team-vulcan">Team Vulcan!</h3>
<p><img src="/static/images/ws-post/2023_04_02_2023_kickoff/IMG_1862.jpg" alt="IMG_1862.JPG"></p>
<h3 id="i-am-canner-i-am-possible">I am Canner, I am possible!</h3>
<p><img src="/static/images/ws-post/2023_04_02_2023_kickoff/IMG_1871.jpg" alt="IMG_1871.JPG"></p>
<h3 id="data-mesh-live">Data Mesh Live!</h3>
<p><img src="/static/images/ws-post/2023_04_02_2023_kickoff/IMG_1881.jpg" alt="IMG_1881.JPG"></p>
<h3 id="four-clouds">Four clouds~</h3>
<p><img src="/static/images/ws-post/2023_04_02_2023_kickoff/IMG_1878.jpg" alt="IMG_1878.JPG"></p>
<h3 id="the-ask-me-trio-see-the-lower-right-corner-of-the-official-website-which-won-a-lot-of-prizes-not-only-looks-alike-but-also-has-interactive-services-which-is-super-exciting">The ASK ME trio (see the lower right corner of the official website), which won a lot of prizes, not only looks alike, but also has interactive services, which is super exciting!</h3>
<p><img src="/static/images/ws-post/2023_04_02_2023_kickoff/IMG_3092_2.jpg" alt="IMG_3092 2.JPG"></p>
<p>Canner is a team of trust, commitment and dedication. Everyone puts their commitment into practice and is fully committed to achieving the company&#39;s goals and vision. At the same time, we also encourage team members to communicate effectively in an independent thinking manner. When we own our ideas and communicate them clearly, we can work together better to create great products and deliver great services. At the same time, we also pay attention to the balance between quality and speed. In our work, we plan our decisions on a user-oriented basis. Our goal is to maximize speed and efficiency while maintaining product and service quality. Moreover, we uphold the philosophy of &quot;starting with the end in mind&quot;, which helps us better manage our time and goals, and better manage our plans and strategies.</p>
<p>In the end, teamwork is the key to success. At Canner, we are united and work together to overcome difficulties and challenges. I am a Canner, I am Possible!</p>


As we begin this dynamic year, we are preparing for new challenges and opportunities. At the Kick-off Meeting, we reflect on the past year's accomplishments, discuss future plans and strategies, and strengthen teamwork and culture.

【Life@Canner】Team Building：2023 Kick-Off Meeting - I am a Canner, I am Possible!

<p>Thank you for joining <code>SELECT * FROM DataMeetup</code> #002 jointly organized by Canner and 源來適你. This event is very special, we are officially authorized by Langchain to hold this Watch Party! At the same time, we invited people in the industry related to LLM/AI and open source to share.</p>
<p>The first session was given by Clkao, who shared with us a lot from his in-depth technical insights and practical application cases. Clkao&#39;s sharing discussed the cutting-edge development and future trends of LLM, giving participants a deeper understanding of the application and advantages of language models.</p>
<img src="/static/images/ws-post/2023_07_27_meetup/pic1.jpeg">
 
<p>For the second session, we invited Cooper, the project leader of <a href="https://getaccio.ai/">Accio</a>. He introduced us to the exciting project of Accio, and joined Ming-Han on the difficulties encountered in the process of integrating with ChatGPT. This sharing is very valuable for people who are exploring similar integrations. The experience sharing of Cooper and Ming-Han saved us a lot of detours.</p>
<img src="/static/images/ws-post/2023_07_27_meetup/pic2.jpeg">
 
<p><code>SELECT * FROM DataMeetup</code> is not only a platform for learning and sharing, but also an opportunity for industry exchanges and cooperation. We have seen scenes where professionals from different backgrounds communicate with each other. </p>
<p>This kind of cooperation and cohesion will undoubtedly bring more innovations and breakthroughs to the AI ​​and open source communities. Thanks to Clkao, Cooper and Ming-Han for their great sharing. Thanks to all the partners who participated in this <code>SELECT * FROM Data Gathering</code>, your enthusiastic participation made this event even more special! We look forward to hosting more events like this in the future and continue to promote the thriving development of the AI ​​and open source community!</p>
<div class="gallery">
     <img src="/static/images/ws-post/2023_07_27_meetup/pic3.jpeg">
     <img src="/static/images/ws-post/2023_07_27_meetup/pic4.jpeg"/>
     <img src="/static/images/ws-post/2023_07_27_meetup/pic5.jpeg"/>
     <img src="/static/images/ws-post/2023_07_27_meetup/pic6.jpeg"/>
     <img src="/static/images/ws-post/2023_07_27_meetup/pic7.jpeg"/>
     <img src="/static/images/ws-post/2023_07_27_meetup/pic8.jpeg"/>
</div>

Thank you for joining 【SELECT * FROM DataMeetup】 #002 jointly organized by Canner and 源來適你. This event is very special. We are officially authorized by Langchain to hold this Watch Party! At the same time, we invited people in the industry related to LLM/AI and open source to share.

【SELECT * FROM DataMeetup】 #002

<blockquote>
<p>Originally post on: <a href="https://vulcansql.com/blog/powering-rapid-data-apps-with-vulcansql">https://vulcansql.com/blog/powering-rapid-data-apps-with-vulcansql</a></p>
</blockquote>
<h2 id="what-are-the-challenges-">What Are the Challenges ?</h2>
<p>When it comes to building data applications on top of your data warehouse, several obstacles can get in the way. Here are the primary obstacles you might come across:</p>
<ol>
<li><p><strong>High Query Latency</strong>:
Customer-facing applications require speedy responses, often within milliseconds. Traditional data warehouses, however, are optimized for analytical workloads, which might lead to slow query responses that could negatively impact your application&#39;s user experience.</p>
</li>
<li><p><strong>Security Concerns</strong>:
In a world where data breaches are all too common, implementing an application-specific security layer is vital. This is especially important for multi-tenant environments where each user must access only their own data.</p>
</li>
<li><p><strong>Cost Considerations</strong>:
The scalability of your applications to serve a large number of concurrent users can bring about cost challenges. As the user base grows, the associated cost of managing a data warehouse might skyrocket. Striking a balance between scalability and cost can be quite a struggle.</p>
</li>
</ol>
<h2 id="meet-vulcansql">Meet VulcanSQL</h2>
<p><strong>VulcanSQL, a data API framework built specifically for data applications</strong>, empowers data professionals to generate and distribute data APIs quickly and effortlessly. It takes your SQL templates and transforms them into data APIs, with no backend expertise necessary.</p>
<p>One of the defining attributes of VulcanSQL is its strategic use of DuckDB&#39;s exceptional capabilities as a caching layer. This synergistic pairing permits VulcanSQL to deliver low-latency APIs, presenting an optimal solution for scenarios where traditional data warehouses may fall short.</p>
<p>By utilizing VulcanSQL, you can move remote data computing in cloud data warehouses, such as Snowflake and BigQuery to the edge. This embedded approach ensures that your analytics and automation processes can be executed efficiently and seamlessly, even in resource-constrained environments.</p>
<p><strong>Visualize a typical use case, where you, a data engineer or analytics engineer, regard the historical data in your data warehouse as &quot;cold data&quot; and store the data frequently accessed or relevant to the application within VulcanSQL as &quot;hot data&quot;.</strong></p>
<p>Now, let&#39;s see how straightforward it is to write a VulcanSQL API.</p>
<h2 id="crafting-a-data-api">Crafting a Data API</h2>
<p>Let&#39;s assume we are creating an API that will provide the daily revenue for the past three years.</p>
<p>For instance, I will set up a VulcanSQL server and use BigQuery as the data warehouse with <strong><a href="http://www.tpc.org/tpch/">TPC-H</a></strong> SF1 data within. We need to handle two things first, but I won&#39;t delve into the specifics here.</p>
<ul>
<li><strong>Project configuration</strong>: Establish project-related configurations. For more information, visit: <strong><a href="https://vulcansql.com/docs/api-plugin/overview">https://vulcansql.com/docs/api-plugin/overview</a></strong></li>
<li><strong>Connecting to a data source</strong>: For more details, visit <strong><a href="https://vulcansql.com/docs/connectors/overview">https://vulcansql.com/docs/connectors/overview</a></strong>. We support BigQuery, Snowflake, PostgreSQL, and Clickhouse.</li>
</ul>
<p><strong>Now let&#39;s dive into the most critical aspect - writing our APIs.</strong></p>
<h3 id="step-1-crafting-apis-with-sql-templates">Step 1. Crafting APIs with SQL Templates</h3>
<p>After <strong><a href="https://vulcansql.com/docs/develop/init">initializing</a></strong> your VulcanSQL project, we can begin constructing your API using SQL templates. Let&#39;s develop a <strong><code>daily_revenue.sql</code></strong> file, which contains the main business logic of the API, all written in SQL.</p>
<p>As an example, I&#39;ll create an API that responds with the daily revenue for a specific period.</p>
<pre><code class="language-sql">{% cache %}
select
  *
from daily_revenue
  where orderdate &gt;= {{ context.params.startdate }}
  and orderdate &lt;= {{ context.params.enddate }}
{% endcache %}
</code></pre>
<ul>
<li>Fetching from Cache: The <strong><code>{% cache %} ... {% endcache %}</code></strong> directive fetches data from the DuckDB caching layer.</li>
<li>Dynamic Parameter: <strong><code>context.params.xxx</code></strong> stands for the values acquired from the query parameters. VulcanSQL uses your template to furnish users with data they specify via parameters.</li>
</ul>
<h3 id="step-2-configuring-your-api">Step 2. Configuring your API</h3>
<p>Next, let&#39;s develop a <strong><code>daily_revenue.yaml</code></strong> file to configure your API.</p>
<pre><code class="language-yaml">urlPath: /daily_revenue
cache:
- cacheTableName: daily_revenue
  sql: &quot;select sum(totalprice), orderdate from cannerflow-286003.tpch_sf1.orders where orderdate &gt;= &#39;1996-01-01&#39; group by orderdate &quot;
  profile: bq
  refreshTime:
      every: &quot;1d&quot;
profile: bq
</code></pre>
<p>Here we&#39;ve defined an API with the path <strong><code>/daily_revenue</code></strong>. In the <strong><code>cache</code></strong> settings, we specific following configurations:</p>
<ul>
<li><strong><code>cacheTableName</code></strong>: The name of the table in DuckDB that will store the cached data.</li>
<li><strong><code>sql</code></strong>: The SQL query that will be executed to extract data from BigQuery.</li>
<li><strong><code>refreshTime</code></strong>: The frequency at which the cache will be refreshed. In this case, it&#39;s daily.</li>
<li><strong><code>profile</code></strong>: We created a BigQuery profile in the project configuration step. Here, we specify the profile to use. Check out the <strong><a href="https://vulcansql.com/docs/connectors/overview">Connecting to Data Sources</a></strong> for more information.</li>
</ul>
<p>Once you start the VulcanSQL server, it syncs data from BigQuery to DuckDB for caching and then serves it at the <strong><code>/daily_revenue</code></strong> endpoint.</p>
<h2 id="performance-evaluation">Performance Evaluation</h2>
<p>In this scenario, we&#39;ve extracted aggregated data from the data warehouse and transformed a SQL template into Data APIs using VulcanSQL. Now, let&#39;s assess the performance of VulcanSQL.</p>
<p>To evaluate if VulcanSQL could serve as a low-latency data API, especially in handling multiple concurrent requests, I conducted a series of load tests using <strong><a href="https://k6.io/">k6.io</a></strong>.</p>
<p>Here&#39;s a look at a basic request. The load test was run for a span of 30 seconds. For a concurrency level of 1 and 10, VulcanSQL managed to handle the load within an impressive 100 milliseconds!</p>
<pre><code># concurrent 1, ran for 30 sec
med 9ms
p(95) 13ms

# concurrent 10, ran for 30 sec
med 48ms
p(95) 71ms
</code></pre>
<p>Even at a concurrency level of 100, VulcanSQL kept the response times within 500ms.</p>
<pre><code># concurrent 50, ran for 30 sec
med 223ms
p(95) 267ms

# concurrent 100, ran for 30 sec
med 422ms
p(95) 492ms
</code></pre>
<p>And with 300 concurrent users, the response time managed to serve within approximately a second, still showing impressive scalability.</p>
<pre><code># concurrent 300, ran for 30 sec
med 1.0s
p(95) 1.3s
</code></pre>
<p>All our source code is available on <strong><a href="https://github.com/Canner/vulcan-sql-examples/tree/main/daily-revenue">Github</a></strong>. Feel free to check it out!</p>
<h2 id="summary">Summary</h2>
<p>In today&#39;s data-driven world, businesses rely heavily on their data warehouses. These rich repositories serve as the backbone for collecting and processing tons of data, turning raw numbers into valuable insights.</p>
<p>However, as more and more organizations look to build data applications atop these data warehouses, a new need arises - the need for a robust data API framework. This framework should be designed to overcome traditional obstacles like high query latency, security concerns, and cost issues, enabling organizations to fully leverage their data and drive their operations to new heights.</p>
<p>Are you ready to step up your data application game with VulcanSQL? Check out our quickstart guide <strong><a href="https://vulcansql.com/docs/get-started/first-api">here</a></strong> and embark on a journey towards more efficient, secure, and cost-effective data applications.</p>
<blockquote>
<p>We have a <a href="https://discord.gg/ztDz8DCmG4">Discord community</a> and welcome to ask us anything or give us feedback! Also, we definitely welcome you to share interesting projects you&#39;ve done using VulcanSQL in <a href="https://discord.com/channels/1017514410677391420/1017514410677391427">#showcase</a> channel!</p>
</blockquote>
<blockquote>
<p>Finally, if VulcanSQL resonates with you, please consider starring us on GitHub!</p>
</blockquote>


Are you building a data application using your data warehouse, be it customer-facing analytics, externally shared APIs, or in-house tools like an admin panel, and finding yourself entangled with latency and cost problems? If so, you've come to the right spot!

Powering Rapid Data Applications Using Your Data Warehouse With VulcanSQL

<p>Amidst the backdrop of Seoul&#39;s towering Lotte Tower, a convergence of innovation, ambition, and dreams unfolded at the K-Startup Grand Challenge 2023 Demo Day. Selected from a staggering pool of over 6,000 global startups, Canner embarked on a transformative journey that led us to this moment, a testament to our commitment and the vibrant potential of the startup ecosystem.</p>
<p><img src="/static/images/ws-post/2023_11_16_k_startup/pic1.png" alt="pic1"></p>
<p>The K-Startup Grand Challenge(KSGC) is not just a government program; it&#39;s a beacon of hope and opportunity for global startups aspiring to delve into the Korean ecosystem. Whether the goal is to raise a new round of investment funds or expand business operations, KSGC stands as a gateway to endless possibilities. Being chosen as one of sixty teams from a massive pool of startups was not just an honor; it was a validation of our vision and hard work.</p>
<p>As we approached the Demo Day, held from October 31st to November 2nd, our objectives were clear. This was our platform to impress a discerning panel of judges, consisting of investors and venture capitalists, who would evaluate our presentations and decide which teams could bring more business, job creation, and innovation to Korea.</p>
<p><img src="/static/images/ws-post/2023_11_16_k_startup/pic2.png" alt="pic2"></p>
<h2 id="the-genesis-of-our-adventure">The Genesis of Our Adventure</h2>
<p>The journey to KSGC began with an intense selection process. Being chosen from over 6,000 startups was exhilarating and overwhelming. We spent countless hours preparing before arriving in Korea, fueled by ambition and a desire to make a mark.</p>
<p>Our aspirations were high. We aimed not only to showcase our startup but also to immerse ourselves in an environment that was a melting pot of innovation and cultural exchange.</p>
<p><img src="/static/images/ws-post/2023_11_16_k_startup/pic3.png" alt="pic3"></p>
<p><img src="/static/images/ws-post/2023_11_16_k_startup/pic4.png" alt="pic4"></p>
<h2 id="south-korea---a-new-chapter">South Korea - A New Chapter:</h2>
<p>Landing in Korea was like stepping into a new world. The initial days were a whirlwind of excitement, learning, and cultural adaptation. The support system provided by KSGC was commendable, offering networking opportunities that were invaluable.</p>
<p>Every day was a new learning experience, from understanding the nuances of the Korean market to interacting with fellow entrepreneurs from around the globe. These moments were not just memorable; they were transformative.</p>
<p><img src="/static/images/ws-post/2023_11_16_k_startup/pic5.jpg" alt="pic5"></p>
<p><img src="/static/images/ws-post/2023_11_16_k_startup/pic6.jpg" alt="pic6"></p>
<h2 id="the-hustle--gearing-up-for-demo-day">The Hustle – Gearing Up for Demo Day:</h2>
<p>Preparing for Demo Day was a journey in itself. It involved rigorous planning, relentless practice, and fine-tuning our pitch. The mentorship and guidance we received were unparalleled, helping us evolve our presentation and strategy.</p>
<p>Our team faced challenges, but our collective spirit and motivation never waned. We supported each other through late nights and early mornings, always focused on the goal ahead.</p>
<h2 id="demo-day--the-pinnacle-of-our-journey">Demo Day – The Pinnacle of Our Journey:</h2>
<p>The atmosphere on Demo Day was electric. The anticipation and energy were palpable as each team prepared to showcase their months of hard work. From the Opening Ceremony to the individual pitching sessions, every moment was charged with potential and possibility.</p>
<p>Our presentation was a culmination of our journey, a story we were proud to tell. We learned about expanding in a foreign ecosystem and received invitations from a few accelerators who were in the audience, seeing great potential in our product for the Korean market.</p>
<p><img src="/static/images/ws-post/2023_11_16_k_startup/pic7.png" alt="pic7"></p>
<p><img src="/static/images/ws-post/2023_11_16_k_startup/pic8.png" alt="pic8"></p>
<p><img src="/static/images/ws-post/2023_11_16_k_startup/pic9.png" alt="pic9"></p>
<h2 id="conclusion">Conclusion:</h2>
<p>As the Demo Day curtains drew to a close, we carried with us more than just the exhilarating experience and the invaluable feedback; we held a renewed sense of purpose and a blueprint for the future. This journey has been a crucible of growth, and we emerge from it not just as contenders, but as a formidable force ready to make our mark on the world. Stay tuned as we turn these lessons into stepping stones towards success.</p>


Amidst the backdrop of Seoul's towering Lotte Tower, a convergence of innovation, ambition, and dreams unfolded at the K-Startup Grand Challenge 2023 Demo Day. Selected from a staggering pool of over 6,000 global startups, Canner embarked on a transformative journey that led us to this moment, a testament to our commitment and the vibrant potential of the startup ecosystem.

Our Journey and Demo Day Experience at the K-Startup Grand Challenge 2023

<p>Happy 2024!</p>
<p>It’s the end of 2023!  I&#39;m so excited and overwhelmed by so many blog posts talking about and mentioning “Semantic Layer” recently…  it&#39;s definitely a fantastic and big year in 2023 for “Semantic Layer”.</p>
<h2 id="the-definite-future">The Definite Future</h2>
<p>Earlier in 2023, there was a <a href="https://www.getdbt.com/blog/dbt-acquisition-transform">big news about dbt&#39;s acquisition</a>  Since then, the data ecosystem has been eagerly waiting for dbt&#39;s next move into the semantic layer. In 2023 October, during the dbt Coalesce, dbt announced its next-generation &quot;<a href="https://www.prnewswire.com/news-releases/dbt-labs-announces-the-next-generation-of-the-dbt-semantic-layer-introduced-alongside-new-integration-with-tableau-301958939.html">dbt semantic layer</a>,&quot; which envisions the future of how the semantic layer looks like. Recently, Jen Grant, the COO of Cube, dubbed 2023 as the year of <a href="https://cube.dev/blog/2023-the-year-of-the-semantic-layer">the semantic layer in a post</a>. She stated that &quot;While Cube and the semantic layer have been around for a long time, it was only in 2023 when the stand-alone semantic layer category turned from a cool idea to a necessary ingredient in any data stack.&quot;</p>
<p>Let’s look at macro statistics, Google Trend also shows starting from 2022, the term “Semantic Layer” started to gain global attention. In 2023, it is at its peak.  2024 will be more exciting for “Semantic Layer”!</p>
<p><img src="/static/images/ws-post/2024_01_02_semantic_layer_vision/g_trend.png" alt="Google Trend 2024"></p>
<h2 id="not-only-us-but-also-asia">Not only US, but also Asia</h2>
<p>Many US-based companies shared customer’s enthusiasm around “Semantic Layer”.</p>
<p>At Canner, we also see the same trend happening not only in the US market but also in Asia.  This year, we are fortunate to provide and implement our solution in banks, insurance companies, manufacturing, gaming, and retail sectors in the Asia region.  By providing Canner Enterprise to our clients we still see  “Semantic Layer” is still in an early stage, and have so much great potential and possibility in the upcoming years.</p>
<p>As many new initiates and innovations are still happening in Canner. Today I would like to share our vision and focus of “<strong>Semantic Layer</strong>” towards 2024.</p>
<h2 id="our-5-key-focus-of-innovating-the-universal-semantic-layer-in-2024">Our 5 Key focus of innovating the Universal Semantic Layer in 2024</h2>
<h3 id="1-query-virtualization-compatibility">1. Query Virtualization Compatibility</h3>
<p>The semantic layer operates as the same fundamental logic across all data applications, including BI, AI, and advanced analytics tools.  The way to achieve this is that Semantic Layer solutions will implement technology like Query Virtualization, which is an essential part of a semantic layer; through virtualization, enterprise data consumers can access data in a vendor-agnostic way; behind the scenes, the semantic layer will automatically rewrite queries and pushdown into source-specific SQL language with optimization and automation.</p>
<p>As the semantic layer grows into mainstream adoption, <strong>the compatibility problem for traditional and latest sources and data applications must be gracefully solved</strong>.  At Canner, I&#39;ve supported many sources and data applications that can connect to the Semantic Layer, sources like Informix, Sybase, Oracle, BigQuery, and applications such as SSIS, SSAS, SAS, Tableau, Power BI, Metabase, etc.</p>
<p>In 2024, we continue to expand the compatibility around data applications and sources.</p>
<h3 id="2-context-aware-metrics-access">2. Context-aware Metrics Access</h3>
<p>The future of metrics, we believe is dynamic and composible, which means that the metrics should be aware of the context. This will allow for the automatic composition of metrics when different users or queries are involved without the need for creating views and tables. To achieve composability, a metrics interface for SQL and APIs is required.</p>
<p><strong>We will need a metrics interface that is a well-defined shared boundary between metrics producers and data consumers. Semantic modeling is defined using the Model Definition Language (MDL), which allows the data team to expose a consistent, strongly typed, and extendable interface</strong>. The client-driven architecture retrieves data tailored to users&#39; needs without knowing any data structure in the metrics. It prunes and filters the correct data out, gratefully decoupling the computation logic and data serving. This enforces data consistency and fast performance by reducing huge chunks of over-fetching data in application tooling.</p>
<h3 id="3-governance-in-business-collaboration">3. Governance in Business Collaboration</h3>
<p>Governance is crucial for any data-related tools used by enterprise customers. The Semantic Layer plays a unique role in the modern enterprise data stack by bridging the gap between data and business teams. This means that many complex governance issues can only be resolved through collaboration between these two teams.</p>
<p>In the future, the governance of the Semantic Layer will be fully integrated with workflow processes. This will include data-sharing processes, approval workflows between departments, and cross-functional teams. The integration will be built on top of basic functions such as auditing trails, granular data access control, lineage, notification, and data policies that will also be incorporated into the Semantic Layer. As a result, teams and departments can collaborate seamlessly without switching between different platforms.</p>
<h3 id="4-enable-ai-in-data-analytics">4. Enable AI in Data Analytics</h3>
<p>According to <a href="https://arxiv.org/pdf/2311.07509.pdf">recent studies</a>, the Semantic Layer can help prevent AI hallucinations. To achieve this, a semantic layer must be established to provide adequate context and semantics to data, as well as define standard data and metric definitions across sources. This will help ensure that the necessary information is provided.</p>
<p>Direct access between AI and raw data can lead to security vulnerabilities. To mitigate this, generating SQL through the semantic layer can ensure granular access control policies are in place.</p>
<h3 id="5-open-source-semantic-layer">5. Open-source Semantic Layer</h3>
<p>We believe that openness is crucial for the ecosystem, and <a href="https://cube.dev/blog/the-need-for-an-open-standard-for-the-semantic-layer">some</a> <a href="https://davidsj.substack.com/p/standard-semantics">people in the community</a> are calling for the semantic layer to have open standards. We plan to open-source our core - the Open Semantic Layer. Our goal is to create query standardization across BI, data science, AI, and advanced analysis tools through open definition standards and query engine. This will help enterprises solve the problem of inconsistency that arises from multiple data definitions in different application tools.</p>
<p>Our plan includes three main design components: <strong>Semantic Modeling, Semantic Definition Language, and Standard Protocol</strong>. Technical engineers can define the Semantic Layer via a syntax similar to GraphQL to define metric definitions. In addition, we provide a self-service UI interface for non-technical personnel to establish semantic standards across different application tools.  We will also provide a standard SQL query via the PostgreSQL wire protocol standard, which most data applications can query against natively and API interface accessible by any application tool, ensuring that all applications can be managed and applied under a unified data governance framework.</p>
<h3 id="hello-2024">Hello! 2024!</h3>
<p>It&#39;s exciting to get started in 2024 and beyond! We will share more details in the upcoming blog posts and sharings, see you there!</p>


Today we would like to share our vision of 2024! 2023 is an incredible year for Semantic Layer, we believe 2024 will be even more exciting!

Canner