1. Strategic thinking

Strategic thinking is the ability to think independently and strategically, to analyze the root cause of a problem and develop solutions. Team members should make decisions based on long-term goals, rather than pursuing short-term benefits.

2. Collaboration

Collaboration involves respecting others' opinions and seeking consensus during discussions. Team members should also help each other complete their work, even under high-pressure situations, while maintaining a calm and clear mindset.

3. Commitment

Commitment means that every team member should take responsibility for the company's performance. They should use smart methods to solve problems, and when mistakes happen, team members should take responsibility together and make quick corrections.

4. Passion

Passion means that every team member should have a passionate and dedicated attitude towards their work. Team members should identify with the company's vision and move towards goals with enthusiasm and the team.

5. Innovation

Innovation means challenging traditional methods and exploring innovative solutions by rethinking the fundamental problems. Team members should be bold in making assumptions and accepting failures in order to create better solutions.

Through these five core values, we believe that every member of Canner can achieve the impossible together.

"I'm a Canner, I'm possible."

In the 2019 Global Competitiveness Report by the World Economic Forum, Taiwan was listed as one of the world's top four innovative countries. How mature is Taiwan's AI implementation in various industries, and what opportunities and challenges do they face? Hive Ventures will release a trend survey on Taiwan's enterprise AI maturity at this summit, revealing the current state of Taiwan's AI layout. Chairman Gong Ming-hsin of the National Development Council, former Google Taiwan Managing Director James Liang, Professor Chien Chen-fu, and renowned Silicon Valley entrepreneur Chen Wu-fu will also participate in discussions with pioneers in the electronics, manufacturing, retail, finance, and healthcare industries to explore Taiwan's AI development, enterprise layout, and future trends.

Canner's data access platform uses data virtualization technology to integrate multi-source databases and cloud-based data, accelerating the production of comprehensive reports and saving up to 70% of data production costs for companies. It is an honor to be able to share and exchange ideas with experts from various fields at the Galaxy Summit 2021, including listed companies in different industries, on the current status of AI implementation in Taiwan's enterprises in 2021.

Canner Enterprise - Data Access Solution for Enterprises.

【SET News】Not just leading in semiconductors! Taiwan companies deploy AI talents to become Asia's leader https://www.setn.com/News.aspx?NewsID=915381 【ET News】Hive Ventures: Taiwan's AI applications expected to catch up with the US within three years https://www.digitimes.com.tw/iot/article.asp?cat=158&cat1=20&cat2=10&id=0000606781_FBD863R37X5P712OQHIPL 【United Daily News】80% of Taiwanese companies are actively introducing AI, but the threshold is "lack of experienced talents" https://money.udn.com/money/story/5612/5341905 【Business Next】80% of Taiwanese companies have implemented AI, a chart shows the progress and challenges they face https://www.bnext.com.tw/article/61947/tw-ai-trend-2021 【China Times】More than 80% of Taiwanese companies have tried to introduce AI and are eager to discover experienced talents https://turnnewsapp.com/ct-featured/223860.html 【Liberty Times】Survey: 80% of companies try to introduce AI, Taiwan-US AI application gap expected to narrow within three years. https://ec.ltn.com.tw/article/breakingnews/3478649?utm_medium=APP&utm_campaign=SHARE&utm_medium=APP

1. Commit and focus

Focus and commitment are what make everything different.

2. Independent thinking, and effective communication

Your unique talent and thoughts are what make you different, make a difference.

We need to create environments in which everyone feels welcomed, fairly treated, and fully supported to do their best. In short, it's about prioritizing mutual respect.

3. Proactive, and action

Do what matters, now.

4. Working backward

Set your end goals first, then weight your decisions.

5. Quality with Speed

Deliver with quality makes things count.

6. Customers first

Happy customers are why a company exists.

7. Team work

Great things in business are never done by one person. They're done by a team of people. — Steve Jobs

“Where exactly am I in that cycle? What have others who have succeeded done? What do I need to make this a successful conquest? Where do I go next?” These questions arise in every board room and meeting we’ve had with large and small enterprises in their quest to transform and optimize their business.

This year, Hive Ventures decided to answer these questions, by embarking on a nationwide survey, to help business decision-makers and solutions providers alike, unveil the mystery behind the current state of enterprise AI adoption in Taiwan.

Get your Free copy today! 👉 https://www.hiveventures.io/sotea

Canner Enterprise - Data Access Solution for Enterprises.

• 【三立新聞】不只半導體領先！台灣企業部署AI人才　成亞洲領頭羊 https://www.setn.com/News.aspx?NewsID=915381
• 【電子時報】蜂行資本：台灣AI應用面3年內有望趕上美國 https://www.digitimes.com.tw/iot/article.asp?cat=158&cat1=20&cat2=10&id=0000606781_FBD863R37X5P712OQHIPL
• 【聯合報】台灣八成企業積極導入AI 但門檻是「缺有經驗人才」 https://money.udn.com/money/story/5612/5341905
• 【數位時代】8成台灣企業已經導入AI，一張圖看懂進度到哪裡！還面臨了哪些挑戰？ https://www.bnext.com.tw/article/61947/tw-ai-trend-2021
• 【中時】台灣已有八成以上企業嘗試導入AI 企業渴望挖掘有經驗人才 https://turnnewsapp.com/ct-featured/223860.html
• 【自由時報】調查：8成企業嘗試導入AI 台美AI應用差距3年內有望追近 https://ec.ltn.com.tw/article/breakingnews/3478649?utm_medium=APP&utm_campaign=SHARE&utm_medium=APP

Leaders have long-term goals and will not sacrifice long-term value for short-term performance. They think from the perspective of the owner. Their every move represents not only their own team, but also the entire company. They never say "that's not my job".

2. Earn Trust

Leaders listen attentively, communicate openly, and respect others. Leaders are brave enough to be self-critical, even if doing so embarrasses or embarrasses them. Not believing themselves or their teams to be always right, leaders hold themselves and their teams to the highest standards.

3. Have Backbone; Disagree and Commit

Leaders should question decisions they disagree with without condescension, even if doing so makes people uncomfortable or exhausting. Leaders should be strong-willed and not easily wavered. They will not easily compromise in order to maintain social cohesion. When a decision is made, they move forward.

4. Think Big, Start Small, and Move Fast

Leaders should formulate and clarify the overall policy and implementation strategy, taking into account bold innovation and inspiration team. Consider issues from different angles, start with the most important things in stages, and execute quickly and accurately.

5. Deliver Results

Leaders will pay attention to the key factors that determine the success or failure of things, and can ensure quality and timely completion. Even if they encounter setbacks, leaders still face them bravely and never give up easily.

What is a Data Fabric?

Conceptually, Data Fabric is an abstraction layer across all the data sources, to empower data teams to collect data information across different sources. Data Fabric is a distributed data environment that connects all data together so that it can be seamlessly accessed by data management services and utilized by end-users, or by applications, and more importantly, data is not physically moved, it is connected by a virtual layer.

Here’s the Data Fabric definition from Gartner.

Source From Gartner Data Fabric Architecture is Key to Modernizing Data Management and Integration

Key features

• Seamlessly integrated with your existing data architecture.

  The Data Fabric must capable of integrating existing data architecture and data sources, by simply connect to the sources. Data sources may be public clouds or on-premises, Data Fabric technology needs to provide a single layer for data access and delivery, this way data silos can be connected through a virtual layer.

• Robust data outputs and destinations

  Through the Data Fabric complex data transformation and optimization, end-users are able to connect to different outputs such as AI, BI, and analytics software.

• Metadata catalog and management

  In Data Fabric software one of the most important features is metadata catalog, which allows data to be searchable, downloadable and all business users can easily access and share between different units and departments.

• Flexible data modeling and transformation

  Using Data Fabric it is also very easy for users to model and transform their data into different dimensions, so different business users are able to inspect and review the data in different aspects and scenarios.

• Data sharing & protection

  Data sharing within an organization and the cross-functional departments is a huge challenge, most enterprises will encounter huge data management bottlenecks while they are scaling their data pipelines. Also, data protection between each user will need a fine-granular permission layer (schema, table, column-level of access control), a built-in user/role-based permission system, and versioning.

Canner

With Canner data access solution, enterprises can connect data silos virtually, and accelerate business data analysis, while ensure data privacy, with data policies.

• Connect with data sources in clicks.
• Optimization for data outputs
• Metadata management
• Data transformation
• Virtual data mart for data sharing & protection.

In 2011, the World Economic Forum (WEF) pointed out in its Global Risk Report that data was going to become the most valuable resource of the 21st century. According to a report from the International Data Corporation (IDC), global data volume is projected to grow to 163 ZB in 2025, showing a tenfold increase from the 16.1 ZB in 2016. The massive amount of data in combination with the rise of big data and data surveying tools mean that uncovering business opportunities from enormous data has already become an essential task for businesses looking to increase their competitiveness. Consequently, this has drawn great focus to data access, otherwise known as data management platforms.

In response to increasing enterprise demand for big data access, many software companies have begun to offer solutions targeting this market, with the most notable among them being the Taiwanese data company Canner Inc. The company was founded in 2016 with an initial focus on developing content management systems but was able to secure investment from the National Development Fund, Hive Ventures, and SparkLabs Taipei in 2018 to develop its Canner Data Access Solution, which was created in response to growing enterprise data surveying demand. The solution currently targets three industries – finance, manufacturing, and retail.

Furthermore, the solution can also compete with foreign options in terms of both price and time by offering much more competitive pricing and allowing projects to be completed within a span of days.

Quick Installation, Affordable Pricing, and Support for Multiple Deployment Modes and Data Connection Sources

Unlike other solutions, Canner only requires a few minutes to install and can easily adapt to various database types, file formats, and cloud/local limitations. The solution can quickly integrate data from different databases and connect isolated data lakes to integrate, convert, synergize, and extricate the truly valuable parts from a sea of data, effectively increasing business competitiveness. Considering the rapid growth of data nowadays, the software also has automated optimization and extensible computation to bypass the lack of relevant manpower in certain businesses, so that even medium-sized businesses can construct their own data platform.

Howard noted that Canner is not only capable of quickly connecting various databases and data stores, but can also be linked to business intelligence (BI) and artificial intelligence (AI) software as well as various data applications to ease program development efforts. Additionally, database administrators can assign necessary access rights based on the jobs and needs of staff in various departments so that every staff member will always have access to data they need, thereby creating the greatest value from data.

Take manufacturing for example, Canner can help businesses rapidly integrate factory information, M&A data, logistic data, and information from both upstream and downstream partners to shorten the time it takes for the product to arrive on the market. For marketing purposes, the solution can be applied to cross-screen, cross-platform, OMO, and O2O data integration. It is worth mentioning that Canner offers flexible installation options to allow businesses to deploy the software in a designated environment according to their preferences and needs. Canner Cloud in particular can be installed within the private cloud of a business to prevent potential data leaks and thereby meet information security and operation demands.

Mentorship from TTA Partner Accelerator Program Helps Accelerate Product Development Cycle

Since Canner Inc. entered the data access market in 2018, the technical prowess of the company has been repeatedly tested and reaffirmed. The Canner virtual Data Access Solution has also gained presence in various industries and increasing renown in Taiwan. This is not only the result of hard work from the team behind the software, but also has a lot to do with capital and technical assistance from the National Development Fund, Hive Ventures, and SparkLabs Taipei.

Howard indicated that one of the key reasons that Canner Inc. products were able to hit the market so quickly is because of assistance from SparkLabs Taipei, an international accelerator program introduced to Taiwan by Taiwan Tech Arena (TTA). The program offers mentorship and advice to help him and his team reduce learning time and overcome challenges. Howard believes that Taiwan is in an excellent position to develop its information technology industry. Canner Inc. will continue to invest in this industry to provide the best virtual data center solution for global businesses.

The data consumption interface is fully compatible with the PostgreSQL wire protocol, which means that it can support any data applications that support PostgreSQL connectors. This makes it possible for businesses to easily access and use data from a variety of sources, regardless of the specific tools or applications they are using.

"We're thrilled to announce these new features for Canner Enterprise," said Howard Chi, CEO of Canner Enterprise. "Our goal has always been to make it as easy as possible for businesses to access and use their data, and these new features are a big step towards achieving that goal. We're confident that our customers will find these new capabilities to be incredibly valuable."

Canner Enterprise's data virtualization offering is designed to help businesses streamline their data management processes, reducing the time and resources required to integrate and use data from multiple sources. With these new features, businesses can more easily access and use data from a wide range of sources, making it easier to gain insights and make informed decisions.

For more information about Canner Enterprise's data virtualization offering, visit the company's website or contact a member of the Canner Enterprise team today at sales@cannerdata.com.

Led by Taiwania Capital, the funding round was also participated in by global venture capital firm Hive Ventures and SparkLabs Taipei, Canner said in a statement.

Canner will use this funding to accelerate product development, expand local and international marketing efforts, and grow its internal teams.

Founded in 2018, Canner aims to empower businesses to efficiently convert data into business value by connecting data silos and transforming business-facing datasets into application-ready dataset APIs with a universal data access interface. With Canner’s data access technology, users can work with datasets without moving or duplicating data between data sources, simplifying the process of building next-generation data applications on top of cloud data warehouses through a universal layer for APIs, access control, data literacy and optimization from diverse data silos.

“Enterprises used to struggle with data silo issues. Even though most data are now centralized in a data warehouse, balancing business innovation and data security remains a big hurdle for organizations due to the complexity and sensitivity around working with data. That is where Canner comes in,

“With Canner’s data access technology, enterprises can virtually enhance their connectivity to data silos and guarantee data security and privacy when collaborating datasets between business units within a company,” said Howard Chi, Co-Founder and Chief Executive Officer of Canner.

Canner’s data access solution can be quickly installed in any cloud – public, private, hybrid and otherwise. It provides multi-format output optimization for data applications, as well as multi-layer data access control and authorization. This allows it to empower organizations to self-serve datasets, achieve data democratization and accelerate innovation, thereby greatly improving enterprise data-driven efficiency.

“The limitations in data availability and accessibility at scale are significant obstacles to enterprise digital transformation. Taiwania believes in Canner’s vision to democratize the emerging data access ecosystem by embracing open-source framework that promises to seamlessly integrate next generation business intelligence applications with analytic tools,” said Cheng Wu, General Partner at Taiwania Capital. An industry veteran, Cheng Wu has founded several high-profile companies with total market value of $6 billion and have led several successful exits.

With its existing support for private and mainstream cloud vendors such as AWS, GCP and Azure, Canner plans to expand beyond its robust local customer base in Taiwan by embracing open-source frameworks and providing Canner cloud Software-as-a-Service (SaaS) services that are available to global customers.

This year, the company also aims to partner with global cloud data warehouse providers such as Redshift, Snowflake, and BigQuery, as well as major solution integrator partners, to accelerate cloud data warehouse and data lake usages across organizations.

“We previously invested in Canner’s seed funding round in 2019 and our participation in this round signifies our continued confidence in their business model. There is no question that data will be the building blocks of future enterprise success, and it is critical that businesses find more efficient ways to access and analyze their insights to enable more agile business transformation. With successful entrepreneurs like Cheng Wu now on the Board of Advisors, we look forward to seeing Canner lead the charge in the data solutions space,” said Yan Lee, Founding Partner of Hive Ventures.

Cheng's entrepreneurship journey began in 1995 when he founded Arris Networks, a startup that developed high-density internet access products that were acquired by Cascade Communications for $217 million in May 1996. In 1997, he founded ArrowPoint Communications and served as CEO, leading the company to a successful IPO. ArrowPoint was acquired by Cisco Systems for $5.7 billion in 2000, and Cheng held various executive positions, including Group VP.

In 2002, Cheng founded Acopia Networks with $65 million, which was later acquired by F5 Networks for $210 million in 2007. That same year, he founded Azuki Systems, specializing in broadband wireless multimedia data transfer software, with a startup capital of $6 million. Azuki Systems was subsequently acquired by Ericsson for $100 million in February 2014.

With his vast experience and expertise in the technology industry, Cheng will provide valuable insights and guidance to Canner. His extensive background in building successful companies will be instrumental in driving Canner's growth and success in the future.

Canner is thrilled to welcome Cheng Wu as an advisor and looks forward to a fruitful partnership with him.

Canner's platform enables businesses and organizations to collect, manage, and analyze data efficiently, and this new deployment support makes it even easier for users to deploy and manage Canner on their preferred cloud platform. Users can now choose between a single VM deployment or Kubernetes deployment to suit their specific requirements.

"We are thrilled to offer our users the ability to deploy and manage Canner on their preferred cloud platform," said the Canner team. "Our platform is designed to be highly flexible and customizable, and this new deployment support is a testament to that."

The new deployment support provides users with a comprehensive guide to deploying Canner on AWS, GCP, and Azure, on single VM or Kubernetes environments. It covers all aspects of the deployment process, including how to set up the environment, how to configure the necessary settings, and how to manage the deployment on an ongoing basis.

To learn more about Canner and its new deployment support, please visit https://cannerdata.com/. The Canner team is always available to provide support and answer any questions that users may have about the platform or the deployment process. With this new deployment support, businesses and organizations can deploy and manage Canner with ease on their preferred cloud platform.

The growth of data innovation has exploded in recent years, mainly due to the thriving cloud data warehouse communities such as Snowflake, Redshift, and BigQuery, with their primary focus on SQL users and business intelligence use cases. Lakehouse architecture brings analytics closer to data lakes, enabling heterogeneous and distributed data processing engines to ingest sources, including diverse workloads such as data science, machine learning capabilities, and near real-time analytics enablement. It has also spawned thriving innovations in integrated data services that automate and unify data modeling, transformation, and metrics, such as dbt and LookML. Collectively, these tools lay the foundation upon which next-generation operational and analytical data applications can be constructed for various data consumers of different personas.

The Data Access Challenges

First of all, let's look at the ultimate goal for data access in a company:

Allow data consumers to find and access their data, fast and simple.

Let's look at a company's current data access workflow; here's what it looks like for a data consumer requesting and accessing data.

1. Data consumers need to find who owns the data.
2. Request access from the data owners; then, they must filter or mask certain rows and columns to specific groups or users before exposing them to usage.
3. Based on different data applications/tools, such as using Excel, BI, AI, or RESTful API, data owners need to evaluate the best data access method.
4. Last is automation. Periodically, update and deliver to end applications and ensure they are secure and auditable.

Data access in an enterprise encounters several challenges.

From left to right, the data heterogeneity needs to homogenize in the metadata and logical level; data authorization is a sophisticated access control and authorization for domain-specific datasets and their associated data applications; empower datasets with semantic meanings through the process of data productization; Finally, based on different data consumers' persona provide endpoints.

A Universal Semantic Layer must solve data heterogeneity, usability, and authorization while enabling consistency and scalability across different data applications.

The Universal Semantic Layer

As companies scale, they can quickly become overwhelmed with data requests, leading to a backlog of demands that can take days or even weeks to resolve. The issue lies in the data access workflow. To overcome this bottleneck, a secure, efficient, and intelligent Universal Semantic Layer is essential, enabling data consumers to access data independently, avoiding delays and misalignment, while ensuring that data owners can authorize and audit datasets, guaranteeing that only the right individuals have access to the appropriate data.

We need a Universal Semantic Layer that is secure, efficient, and intelligent.

Ultimately, achieve equilibrium between data, people, and applications.

Four key designs

Canner Enterprise centralizes governance and control across data applications, allowing for unified access to diverse data sources. We offer a more efficient and effective way to manage and govern data, reducing complexity and risk while providing greater transparency and control.

1. Data Virtualization

Universal Semantic Layer is collaborative and distributed in nature, with each silo or data source independently scalable or together as an aggregate.

2. Data Product Management

Transform data models to domain-oriented datasets; Domain-oriented datasets owned by data owners can be shared and governed by open APIs, with the flexibility of interchangeable metadata and access rules, let data speak your business language.

3. Data Security

Consistent data authorization framework from sources to data applications and integrated with existing Identity and Access Management (IAM). Make data authorization consistent across data sources, IAM, and data applications.

4. Data Serving

Data consumers can generate Queries and APIs with intent and contextual settings, applied to the corresponding datasets via intent declaration, and deliver them to target consumers where final analytics are performed and displayed.

The Result

Enterprises can significantly eliminate data complexity, communication, and productivity through the Universal Semantic Layer.

1. Data access from days to minutes: Reduce 60% of data integration cost with up-to-date data delivery.
2. Reduce duplicate datasets: Create masked and filtered datasets without physically moving data.
3. Achieve self-service analytics: Improve data productivity across analytical and operational data applications and tools.

Canner Enterprise is a powerful data access platform that provides manufacturers with a unified view of their data from multiple sources, regardless of its format, location, or structure. With Canner Enterprise, manufacturers can easily access and integrate data from various systems, such as ERP, CRM, MES, among others.

Manufacturers face numerous challenges when it comes to data management. For example, in a highly competitive market, manufacturers must optimize their operations to reduce costs and improve productivity continually. Canner Enterprise can help manufacturers access and analyze real-time data from various production systems, enabling them to identify inefficiencies, bottlenecks, and other issues that impact productivity.

Today’s data challenge

Manufacturing without data virtualization can be challenging and inefficient, as manufacturers face several pain points related to data management and access. Here are some of the most significant pain points of manufacturing without data virtualization:

1. Data silos: Manufacturing systems generate vast amounts of data, which often gets stored in isolated silos, making it challenging to access and analyze. This can lead to a fragmented view of operations, and manufacturers may miss valuable insights and opportunities for optimization.
2. Complex integrations: Integrating data from various systems can be a complex and time-consuming process, especially if the systems use different formats or protocols. Manufacturers may need to develop custom integration solutions or rely on manual data transfers, which can be error-prone and inefficient.
3. Data duplication: In many cases, manufacturers need to replicate data across multiple systems to ensure all teams have access to the necessary information. This can lead to data duplication, inconsistencies, and errors, making it difficult to trust and rely on the data.
4. Limited access: Data may be inaccessible to some teams or stakeholders, leading to siloed decision-making and missed opportunities for optimization. In some cases, manufacturers may need to provide physical access to data centers or require specialized skills to access the data, limiting its usability.
5. Slow data retrieval: Without data virtualization, retrieving data from various systems can be slow and time-consuming. Manufacturers may need to perform multiple queries or wait for manual data transfers, slowing down decision-making and hindering operational efficiency.

Scenarios of using Data Virtualization in manufacturing:

Data virtualization is a technology that allows businesses to access and integrate data from multiple sources without having to physically move or replicate the data. In the manufacturing industry, data virtualization can be used in a variety of ways to improve data access, integration, and analysis. Here are some scenarios of using data virtualization in manufacturing:

1. Supply chain management: Manufacturers can use data virtualization to integrate data from various sources such as suppliers, logistics providers, and internal systems to gain a more comprehensive view of their supply chain. By virtualizing data, they can quickly and easily access data in real-time, perform simulations and analyses, and make more informed decisions.
2. Production planning: Manufacturers can use data virtualization to access and integrate data from various production systems, including machine sensors, shop floor systems, and quality control systems. By virtualizing data, they can quickly analyze production data, identify patterns, and optimize production processes.
3. Quality control: Manufacturers can use data virtualization to integrate data from various quality control systems, including production systems, supplier systems, and customer feedback systems. By virtualizing data, they can quickly identify quality issues, track quality metrics, and make improvements to their quality control processes.
4. Asset management: Manufacturers can use data virtualization to integrate data from various asset management systems, including machine sensors, maintenance systems, and inventory systems. By virtualizing data, they can quickly analyze asset performance, track maintenance requirements, and optimize asset utilization.
5. Sales and marketing: Manufacturers can use data virtualization to integrate data from various sales and marketing systems, including customer databases, market research systems, and social media feeds. By virtualizing data, they can quickly analyze customer behavior, identify trends, and improve sales and marketing strategies.

Fortunately, there is a solution that can simplify data management and access for financial institutions - data virtualization. And Canner Enterprise, the data access layer, is at the forefront of this technological revolution.

Canner Enterprise — Data Access Platform

Canner Enterprise is a powerful data access platform, that leverage data virtualization technology that allows financial institutions to access and manage data from multiple sources, regardless of the data's format, location, or structure. This means that financial institutions can easily access data from various systems, such as core banking, CRM, risk management, and compliance systems.

With Canner Enterprise, financial institutions can solve many financial problems, such as financial holding, insurance, bank, securities, and more. For instance, in financial holding companies, data is scattered across various business units and systems, making it challenging to gain a complete view of the organization's operations. Canner Enterprise can provide a unified view of the organization's data, enabling managers to make better decisions and identify potential risks and opportunities.

In the insurance sector, data is crucial for risk assessment, underwriting, and claims management. Canner Enterprise can provide insurance companies with a 360-degree view of their customers, policies, and claims, enabling them to provide personalized services and streamline claims processing.

For banks, compliance is a critical issue. Canner Enterprise can help banks access and manage data from various sources, such as anti-money laundering systems, fraud detection systems, and transaction monitoring systems, among others. This can help banks comply with regulations, detect suspicious activities, and prevent fraud.

In the securities industry, data is crucial for investment decisions, risk management, and portfolio optimization. Canner Enterprise can help securities firms access and analyze data from various sources, such as market data providers, research firms, and trading systems. This can help securities firms make informed investment decisions and manage risks effectively.

Scenarios of using Data Virtualization in finance:

1. Risk management: Banks and financial institutions can use data virtualization to access and integrate data from multiple sources such as credit bureaus, regulatory agencies, and internal systems to gain a more comprehensive view of their risk exposure. By virtualizing data, they can quickly and easily access data in real-time, perform simulations and stress tests, and make more informed decisions.
2. Customer analytics: Financial institutions can use data virtualization to combine data from customer touchpoints such as customer service systems, transaction systems, and marketing databases to gain a more comprehensive view of their customers. By virtualizing data, they can quickly and easily analyze customer behavior, identify patterns, and improve customer engagement.
3. Compliance reporting: Financial institutions can use data virtualization to access and integrate data from various systems and sources to ensure compliance with regulations. By virtualizing data, they can quickly generate compliance reports, monitor and audit transactions, and provide real-time access to regulatory data.
4. Portfolio management: Investment firms can use data virtualization to access and integrate data from various sources, such as market data feeds, research reports, and internal systems to make more informed investment decisions. By virtualizing data, they can quickly analyze market trends, assess risks, and optimize their portfolios.
5. Fraud detection: Financial institutions can use data virtualization to access and integrate data from various systems such as transaction, account opening, and fraud detection systems to detect and prevent fraudulent activity. By virtualizing data, they can quickly identify patterns, track suspicious activity, and prevent fraud before it occurs.

Canner has always been committed to pursuing a work style that is most friendly and respectful to all team members. After a period of experimentation and trial, we have developed Canner's unique "hybrid remote work model" where team members come to the office twice a week.

Going in office two days a week.

During these office days, we have meetings where we discuss important information and brainstorm ideas face-to-face. We also have casual conversations over lunch or afternoon tea, which is a great way to build relationships and foster a positive work environment. Changing the work environment can also stimulate new ideas and creativity.

On the other days, team members work remotely from home. This saves commuting time and provides flexibility in choosing a work location, whether it be in the city or in their hometown. We also use Gather, a virtual meeting space, to connect with remote team members and have discussions or brainstorming sessions.

To ensure effective communication and team cohesion, Canner has established guidelines for remote work. These guidelines include Text Discuss, Video Discuss, Transparent Discussion, and Proactive Check. By following these guidelines, we can avoid communication difficulties and conflicts.

Remote culture is built on trust among team members. Even though we work remotely, our work quality is not affected. At Canner, we work in a casual but responsible manner, and we believe that our unique approach to remote work is a key factor in maintaining positive relationships within the team.

PostgreSQL wire protocol is important in the data ecosystem for several reasons:

1. Client-Server Communication: The PostgreSQL wire protocol defines the rules for communication between a PostgreSQL client and server. This is important because it allows applications and tools to connect to and interact with the PostgreSQL database, retrieve data, and perform database operations.
2. Standardization: The PostgreSQL wire protocol is a standard, which means that it can be used by different tools and applications that support the protocol. This ensures that the data can be accessed and processed in a consistent and standardized way across different systems.
3. Security: The PostgreSQL wire protocol provides a secure way to transmit data between the client and server by using SSL/TLS encryption. This ensures that sensitive data is protected during transmission, and makes it less vulnerable to interception or attacks.
4. Performance: The PostgreSQL wire protocol is designed to be efficient, which means that it can transmit data quickly and with minimal overhead. This is important for applications that need to process large amounts of data quickly.
5. Extensibility: The PostgreSQL wire protocol is extensible, which means that new features can be added to it as needed. This allows the protocol to evolve over time and support new use cases and requirements.

The adoption rate of PostgreSQL wire protocol

It's difficult to determine the exact adoption rate of PostgreSQL wire protocol because it is a widely used protocol and there is no central repository or authority that tracks its usage. However, we can make some general observations about its adoption based on various sources of information.

According to the DB-Engines ranking, PostgreSQL is the fourth most popular database management system as of February 2023. Another indication of the adoption rate of the PostgreSQL wire protocol is the number of tools and applications that support it. Many popular tools and frameworks, such as JDBC, ODBC, provide support for the PostgreSQL wire protocol, making it accessible to a wide range of developers.

Databases based on PostgreSQL wire protocol

There are several databases that are based on the PostgreSQL wire protocol. Some of the most popular ones include:

1. Amazon Redshift: Amazon Redshift is a data warehousing service based on PostgreSQL wire protocol. It is designed to handle large-scale data analytics workloads and provides a scalable and cost-effective solution for processing and analyzing large amounts of data.
2. YugabyteDB: YugabyteDB is a distributed SQL database that supports PostgreSQL wire protocol. It is designed to provide high scalability, fault tolerance, and consistency, making it a popular choice for modern cloud-native applications.
3. Citus: Citus is a distributed database that extends PostgreSQL with features such as horizontal scaling and sharding. It supports PostgreSQL wire protocol and is designed to handle large-scale, real-time workloads.
4. CockroachDB: CockroachDB is a distributed SQL database that supports PostgreSQL wire protocol. It is designed to provide high availability, scalability, and performance, and is often used for modern cloud-native applications.
5. TimescaleDB: TimescaleDB is a time-series database that extends PostgreSQL with features such as time-based partitioning and indexing. It supports PostgreSQL wire protocol and is designed to handle large-scale, time-series data workloads.

These databases are based on PostgreSQL wire protocol, which means that they can be accessed and used using the same tools and applications that support PostgreSQL. This makes them a popular choice for organizations that are already familiar with PostgreSQL and want to extend its functionality or scalability.

Canner Enterprise adopt the postgresql wire protocol

Canner Enterprise leverages PostgreSQL wire protocol to bring the best connectivity to end applications, such as BI, AI, and data analytics software, Canner Enterprise allows you to create a virtual data access layer that sits between your application and data sources, enabling real-time access to data across multiple sources without the need for data movement or replication.

This approach can significantly reduce the complexity and cost of managing data integration and provides a more flexible and scalable way to access and analyze data. With Canner Data Virtualization, you can easily connect to a wide range of data sources, including cloud-based and on-premises databases, and create a unified view of data that can be easily accessed by your applications. Whether you're dealing with big data, real-time data processing, or complex data integration requirements, Canner Data Virtualization can help you streamline your data architecture and improve your data access and analysis capabilities.

Data Virtualization

Data Virtualization is an enterprise integration technology that distributes data across different systems. It provides a unified data view through a virtual layer and can integrate data into multiple applications. Data Virtualization technology abstracts different data sources, allowing enterprises to easily access data without worrying about where it is stored. Using Data Virtualization technology can also increase data availability and integrity since data is integrated into a single view.

Data Fabric

Data Fabric is an enterprise integration technology that also distributes data across different systems, but it places more emphasis on the overall structure of data. Data Fabric technology allows enterprises to move and manage data freely between different data sources. It can provide a unified data management platform for enterprises, thereby increasing data availability and reliability. Using Data Fabric technology can also increase enterprise control over data and enhance data security.

Differences between Data Virtualization and Data Fabric

Data Virtualization is a transparent way of integrating distributed data. It typically integrates multiple data sources (such as relational databases, NoSQL databases, web services, and other data sources) into a virtual database, allowing users to access data from all data sources through a single interface. This provides a unified way of accessing data, reduces the need for data replication and synchronization, and improves the efficiency of data integration.

Data Fabric is a more comprehensive architectural model that aims to achieve an overall structure for enterprise data. Data Fabric usually refers to a unified data structure that can integrate different data sources and provide users with a unified data access interface. The goal of Data Fabric is to provide an overall data structure that makes it easier for enterprises to manage and control data, as well as to facilitate data analysis and application development.

In short, Data Virtualization focuses more on data integration technology, with a focus on achieving transparent integration of distributed data; while Data Fabric focuses more on data integration architecture, with a focus on achieving an overall structure for enterprise data that makes it easier for enterprises to manage and control data.

Implementing Data Fabric with Canner Enterprise

Data Virtualization technology and Data Fabric architectual concept are important innovation for modern enterprises in data integration. Canner Enterprise designs highly scalable and supportive Data Virtualization technology to assist enterprises in implementing Data Fabric architecture.

Importance of Data Virtualization in Enterprises

Data virtualization has many benefits. Firstly, it can help businesses better manage data, thus increasing business efficiency. By integrating data from different sources into a virtual database, businesses can easily access and analyze this data without the need for complex data transformation and integration work. This can save a lot of time and manpower, thereby increasing the efficiency and competitiveness of businesses.

Here are several key applications in enterprises:

1. Simplify data integration: Different departments and systems within an enterprise may have unique data formats and structures, making data integration complex and time-consuming. Data virtualization can integrate and consolidate data without duplicating it, simplifying the data integration process.
2. Enhance data availability: Through data virtualization, businesses can centralize data from different sources into a virtual data layer, enhancing data availability and accessibility. This enables businesses to use and share data more easily, thereby increasing productivity and improving the quality of decision-making.
3. Improve data security: Data virtualization can help businesses keep data in the source system, reducing the risk of data leaks and security issues. Additionally, by setting permissions and controlling access, businesses can better protect their sensitive data.
4. Optimize data management: Data virtualization can help businesses better manage their data, improving data quality and reliability. Through data virtualization, businesses can organize and manage their data without disrupting existing data structures and systems.
5. Reduce costs and risks: Through data virtualization, businesses can reduce the cost of data integration and management. Additionally, as data virtualization does not require data to be copied and transferred between different systems, it can lower the risk of data loss and inconsistency.

Future of Data Virtualization

Data virtualization will have broader applications in the future. Firstly, as artificial intelligence and big data technologies develop, businesses will need more data for analysis and prediction. Data virtualization can help businesses better manage this data, thereby increasing their predictive and decision-making capabilities. For example, businesses can use data virtualization technology to integrate data from different sources into a virtual database and then use artificial intelligence technology to analyze and predict this data, resulting in more accurate results.

1. Big data and AI: Data virtualization can help businesses better manage and integrate big data, improving the application of data analysis and AI technology. Through data virtualization, businesses can easily integrate data from multiple sources into a virtual data layer, creating larger datasets and achieving more accurate analysis and prediction.
2. Cloud data and multi-cloud architecture: As businesses and organizations increasingly adopt cloud data and multi-cloud architecture, data virtualization will become more important. Data virtualization can help businesses integrate data from different cloud platforms into a unified data layer, thereby improving data accessibility and availability.
3. Data privacy and compliance: As data privacy and compliance issues become more important, data virtualization can help businesses better protect sensitive data and ensure the legality of data use. By setting permissions and controlling access, businesses can better control data access and use, thereby ensuring data privacy and compliance.
4. IoT and edge computing: As the application of the Internet of Things (IoT) and edge computing technologies becomes more widespread, data virtualization will play an important role. Data virtualization can help businesses better manage and integrate data dispersed across different devices and edge computing nodes, achieving more effective data analysis and prediction.

Canner Enterprise Data Virtualization Technology

Data virtualization in Canner Enterprise is an important new technology that can help businesses better manage and analyze data, improving predictive and decision-making capabilities. In the future, data virtualization will have broader applications, helping businesses address various challenges and opportunities. Therefore, businesses should strengthen their learning and application of data virtualization technology, thereby improving their competitiveness and innovation.

Effective team collaboration is one of the important keys to a company's success and continued growth. Through fun Team Building activities, team members can step out of their daily office environment, think outside the box, and not only improve communication among team members, but also enhance their ability to think from others' perspectives and work together as a team.

Besides, during the busy work schedule and the mask-wearing situation in the office, we have fewer opportunities to communicate with colleagues from different teams, which can make us feel like familiar strangers.

As the pandemic situation gradually improves, to promote interactions between different teams, we decided to have an Outing Team Building activity in the beautiful nature of Yilan. We planned a series of indoor and outdoor activities, allowing everyone to relax a bit from their busy work and get to know colleagues from different teams.

The first stop away from the office was in Dong'ao, Yilan, where we tried the popular SUP (Stand-up paddleboarding) activity. Before getting into the water, everyone was excited!

It turned out that the SUP activity required teamwork and cooperation. Only when everyone worked together and coordinated with each other could we form a line of paddleboards. We also needed to take care of colleagues who were not good at water activities, so that they could have fun and feel safe.

In this Outing, we not only cultivated better team rapport but also rewarded everyone for their hard work over the year. Well done!

The brain-teasing team games in Team Building not only tested our IQ but also our teamwork skills XD

After a night's rest, we continued our adventure along the Nan'ao coastline, riding ATVs to enjoy the scenery and clear our minds.

From not being able to know everyone's name within the first month of joining the company to immediately calling out the correct name after two days and one night of team building, that's the magic of actual interactions! Apart from consolidating team cohesion, we gained many benefits, such as:

Stepping out of the comfort zone and discovering different aspects of ourselves and others

Everyone has their own habits and preferences. Usually, during leisure time, we choose activities that we enjoy and are good at. However, in Team Building, we do activities that we may not normally engage in and talk to people from different backgrounds, which can inspire different ideas or find common topics.

Taking a break and coming back with a better self

As mentioned at the beginning of the article, we sometimes face obstacles that are difficult to overcome at work and get stuck in an infinite loop. Putting down the computer and taking a break, chatting aimlessly with team members, and looking at the boundless sea view can recharge us, and we can return to work with a better self and continue to strive!

Creating a group memory

A journey is made up of many exciting memories. A memorable activity adds a common memory to the entire team. During breaks, we occasionally talk about the muscle soreness caused by a large amount of outdoor activities, but we also remember the feeling of unity and laughter during the activities. With this memory that belongs only to us, our relationship has improved even more.

In addition to static photos for memories, dynamic videos can also bring you an immersive experience

Analytics APIs are the primary interface for data consumers to utilize data in their daily business applications, such as BI, reports, dashboards, spreadsheets, and web applications. However, data stored in data warehouses are not accessible for those users and tools without an API consumption layer. VulcanSQL aims to solve that problem by translating SQL into flexible APIs. It is contextual in that it can translate APIs into the corresponding SQL based on different user personas and business contexts.

With VulcanSQL, data engineers can quickly and easily create APIs without having to write any backend code. This makes it possible to accelerate development time and reduce the complexity of building data APIs. The generated APIs are highly scalable and performant, making it possible to serve thousands of requests per second without any degradation in performance.

VulcanSQL is designed to be highly customizable and extendable. It comes with built-in support for complex SQL queries and custom business logic, making it possible to create highly tailored APIs that meet specific business requirements. This flexibility makes it possible to create highly sophisticated data API layers that integrate seamlessly with existing business applications and data warehouses.

• Github: https://github.com/Canner/vulcan-sql
• Website: https://vulcansql.com
• Discord: https://discord.gg/8JAsVFfNes

The popularity of modern data stack is related to the modern enterprise's need to process large amounts of data from multiple sources, including websites, mobile applications, social media, sensors, servers, and other databases. Furthermore, businesses need to transform this data into useful insights to make better business decisions.

Traditional data stacks typically require the use of multiple different tools and technologies to complete various stages of data processing. These tools are often incompatible with each other and require a lot of manual configuration and maintenance. On the other hand, modern data stacks are more automated and scalable, usually composed of tools and platforms that can be integrated and upgraded more easily, and generate useful data analysis insights more quickly.

Therefore, modern data stack has become increasingly popular in recent years, especially in large and data-driven enterprises that need to process large amounts of data to improve their business efficiency and competitiveness.

What is Modern Data Stack?

Modern Data Stack refers to a set of modern technologies, tools, and processes that work together to help businesses collect, store, process, and analyze data in real-time, enabling data-driven decision-making.

The modern data stack includes several components, such as:

1. Data sources: These are the systems or platforms where data is generated or collected, such as databases, applications, APIs, and IoT devices.
2. Data pipelines: These are the tools and processes used to extract data from the data sources, transform it into a usable format, and load it into a data warehouse or data lake. Examples of data pipeline tools include Apache Kafka, Apache NiFi, and Fivetran.
3. Data warehouse or data lake: These are the storage systems where data is stored and made accessible for analysis. Examples of data warehouse systems include Amazon Redshift, Google BigQuery, and Snowflake. Examples of data lake systems include Apache Hadoop and Amazon S3.
4. Business intelligence and analytics tools: These are the tools used to explore and analyze data, create dashboards and reports, and gain insights from data. Examples of BI and analytics tools include Tableau, Looker, and Power BI.
5. Data governance and security tools: These are the tools used to ensure that data is properly secured, comply with regulations, and manage access to data.

Why is Modern Data Stack so popular?

With the explosive growth of data, enterprises need more data analysis and data science support to help make better decisions. Traditional data analysis processes often require a lot of time and manpower, especially in data extraction and transformation, which often requires technical assistance. The emergence of the Modern Data Stack allows non-technical personnel to easily perform data analysis, thereby improving the efficiency of data analysis and saving costs.

In addition, Modern Data Stack has the following advantages:

1. Scalability: Modern data stack technologies are designed to scale seamlessly as data volume and complexity grow, enabling businesses to process and analyze vast amounts of data quickly and easily.
2. Real-time data processing: The modern data stack enables businesses to process and analyze data in real-time, providing up-to-the-minute insights and enabling faster decision-making.
3. Integration: The modern data stack is designed to be highly integrated, enabling businesses to easily connect data sources, data pipelines, and analytics tools.
4. Democratization of data: The modern data stack makes it easier for businesses to share data and insights across teams and departments, enabling a more data-driven culture and empowering individuals to make data-driven decisions.
5. Flexibility: The modern data stack is designed to be flexible, enabling businesses to easily add or remove components as their needs change and new technologies emerge.

Why should enterprises pay attention to Modern Data Stack?

1. Competitive advantage: The modern data stack enables businesses to extract valuable insights from their data, providing a competitive advantage in a rapidly changing business landscape.
2. Improved decision-making: By leveraging the modern data stack, enterprises can make data-driven decisions quickly and effectively, leading to better business outcomes.
3. Increased efficiency: The modern data stack automates many of the data processing and analysis tasks, enabling businesses to be more efficient and productive.
4. Better customer experiences: The modern data stack enables businesses to better understand their customers, leading to more personalized and targeted experiences that drive loyalty and retention.
5. Compliance and governance: The modern data stack includes tools for compliance and governance, ensuring that enterprises remain compliant with regulations and protect the privacy of their customers.
6. Innovation: The modern data stack enables enterprises to experiment with new data sources, analysis techniques, and technologies, leading to new insights and opportunities for innovation.

Using Canner as a Data Access Layer of the Modern Data Stack

Enterprises can benefit from data access layers in many ways, including:

1. Improved data quality and consistency: The data access layer can help ensure data consistency and accuracy in different tools and applications, reducing the risk of errors and inconsistencies in data analysis.
2. Increased production efficiency: The data access layer provides a unified interface for accessing and analyzing data, reducing the need for redundant data processing and improving the efficiency of data analysis. This can increase productivity and reduce costs.
3. Enhanced security: The data access layer can help ensure secure and controlled use of data, minimizing the risk of data leakage and unauthorized access to sensitive data.
4. Scalability: The data access layer is designed to be highly scalable, able to handle large amounts of data and support multiple concurrent users. This can help enterprises expand their data capabilities over time as needed.
5. Flexibility: The data access layer can be customized according to the specific needs and requirements of the enterprise, allowing the organization to use its data in ways that suit its unique needs and goals.
6. Better decision-making: By providing a unified interface for accessing and analyzing data, the data access layer can help enterprises generate insights more quickly and make more informed, data-driven decisions.

1. Data security: One of the main challenges of moving data to the cloud is ensuring data security. The data must be protected from unauthorized access during transit and at rest in the cloud. In addition, there must be measures in place to ensure data privacy and compliance with regulations.
2. Data integration: Data integration can be challenging when moving data from an on-premise environment to the cloud. The data may be stored in different formats, and it may be difficult to integrate the data with other systems in the cloud.
3. Data latency: Data latency can be an issue when moving data to the cloud. The distance between the on-premise environment and the cloud can cause delays in data transmission, which can impact application performance.
4. Data governance: When moving data to the cloud, it is important to maintain policies to ensure data is used appropriately. This can include access controls, auditing, and compliance monitoring.

How Data Fabric Addresses Challenges in Moving Data to the Cloud

Data fabric can help address these challenges by providing a unified, virtualized view of the data that spans both on-premise and cloud environments. Data fabric enables data to be accessed and integrated from multiple sources, regardless of where the data is stored. This can help address data integration challenges by providing a single view of the data, regardless of where it is stored. In addition, data fabric can help address data latency challenges by caching frequently accessed data in memory, reducing the need to access the data across the network.

Data fabric also provides a layer of abstraction that can help address data security and governance challenges. The data can be protected and managed in a consistent way, regardless of where it is stored. This can help ensure data security and compliance with regulations.

Real-world examples

here are some real-world examples of how data fabric is being used to address the challenges of moving data from on-premise environments to the cloud:

1. Financial services: A global bank wanted to move their data from an on-premise environment to the cloud to reduce costs and improve scalability. However, they faced challenges with data integration and security. By using a data fabric platform, the bank was able to create a unified view of their data that spanned both environments, enabling them to easily move their data to the cloud while maintaining data security and compliance.
2. Healthcare: A healthcare provider wanted to move their data to the cloud to improve collaboration and analytics. However, they faced challenges with data latency and governance. By using a data fabric platform, the provider was able to create a unified view of their data that spanned both on-premise and cloud environments, enabling them to access their data in real-time and maintain data governance policies across both environments.
3. Retail: A large retailer wanted to move their data to the cloud to improve customer insights and personalization. However, they faced challenges with data integration and latency. By using a data fabric platform, the retailer was able to create a unified view of their data that spanned both environments, enabling them to easily integrate their data and access it in real-time, improving their customer insights and personalization capabilities.
4. Manufacturing: A manufacturer wanted to move their data to the cloud to improve supply chain management and analytics. However, they faced challenges with data integration and security. By using a data fabric platform, the manufacturer was able to create a unified view of their data that spanned both environments, enabling them to easily integrate their data and maintain data security and compliance policies.

Canner is a powerful data access platform that enables businesses and organizations to collect, manage, and analyze data efficiently. With this new documentation support in both English and Traditional Chinese, users will have the opportunity to learn more about Canner and how to use its powerful features and functionalities in their preferred language.

"We understand the importance of providing our users with comprehensive support in their preferred language," said the Canner team. "Our goal is to make Canner's platform accessible to users all around the world, and this new documentation support is a major step towards achieving that goal."

The new documentation support provides users with a comprehensive guide to using Canner's platform. It covers all aspects of the platform, including how to get started, how to set up data sources, how to manage data, and how to analyze data using Canner's powerful tools and features. The documentation also includes helpful tips and best practices for using Canner's platform effectively.

To learn more about Canner and its new documentation support in English and Traditional Chinese, please visit https://docs.cannerdata.com. The Canner team is always available to provide support and answer any questions that users may have about the platform.

DBLink is a feature in many relational database management systems (RDBMS) that allows tables in one database to be accessed from another database. DBLink works by establishing a connection between two databases, enabling SQL statements to be executed across the two databases. DBLink is primarily used for accessing data within RDBMS systems, and it requires both databases to be running on the same network.

Data virtualization, on the other hand, is a technology that allows data to be accessed and integrated from multiple sources, regardless of where the data is located. Data virtualization provides a virtualized view of the data, which enables users to access and analyze the data without physically moving or replicating the data. Data virtualization works by creating a virtual layer that integrates data from multiple sources and presents it as a single view, which users or applications can access.

The key difference between DBLink and data virtualization is that DBLink is limited to accessing data within RDBMS systems, while data virtualization can be used to integrate data from any source, including RDBMS systems, NoSQL databases, cloud storage systems, and APIs. Data virtualization also provides a more flexible and scalable solution for data integration, as it does not require data to be moved or replicated, and it can be used to access and integrate data in real-time.

What are the problems of using DBLink

While DBLink can be a useful feature for accessing data across multiple databases within a relational database management system (RDBMS), there are several challenges and limitations to using DBLink. Here are some of the problems of using DBLink:

1. Performance: DBLink can impact system performance when accessing data across different databases, as it involves additional network traffic and processing overhead. This can lead to slower query response times and impact the performance of the database system.
2. Security: When using DBLink, users need to have the appropriate permissions to access the data in the remote database. This can raise security concerns, as it involves granting users access to data in another database.
3. Compatibility: DBLink may not be compatible with all RDBMS systems, which can limit its usefulness for organizations using multiple databases.
4. Maintenance: DBLink requires ongoing maintenance and management, including monitoring for errors and issues, and updating configurations as needed.
5. Scalability: DBLink may not be scalable for larger data sets or for organizations with complex data integration requirements. As the number of databases and tables increases, managing DBLink connections can become more complex.

Organizations need to carefully consider the performance, security, compatibility, maintenance, and scalability implications of using DBLink before implementing it. Alternative solutions, such as data virtualization, may offer a more flexible and scalable approach for data integration that addresses some of the limitations of DBLink.

Can data virtualization help?

Data virtualization can be a useful solution for addressing some of the problems associated with using DBLink. Here's how:

1. Performance: Data virtualization can improve query performance by caching frequently accessed data in memory, reducing the need for data to be accessed across the network. This can improve query response times and reduce the impact on the performance of the database system.
2. Security: Data virtualization provides a layer of abstraction that can help address security concerns by managing and controlling data access in a consistent way, regardless of where the data is located. This can help ensure that data is accessed only by authorized users, and that data privacy and compliance requirements are met.
3. Compatibility: Data virtualization is designed to be compatible with a wide range of data sources, including relational and non-relational databases, cloud storage systems, and APIs. This provides a more flexible and scalable solution for data integration than DBLink.
4. Maintenance: Data virtualization can help reduce maintenance requirements by providing a centralized view of data, eliminating the need to manage and maintain multiple connections to different databases. This can help reduce the complexity and cost of managing data integration.
5. Scalability: Data virtualization can help improve scalability by providing a more flexible and scalable solution for data integration. As the number of data sources and tables increase, data virtualization can help manage the complexity of data integration and provide a more scalable solution.

With Canner Enterprise, the Semantic Layer

Canner Enterprise’s Semantic Layer can help address the challenges associated with using DBLink, providing a more flexible and scalable solution for data integration that can help improve query performance, data security, compatibility, maintenance, and scalability. By using Caner Enterprise, organizations can create a unified, virtualized view and metrics of their data that spans multiple sources, enabling them to make more informed decisions based on a more comprehensive view of their data.

In a traditional centralized data architecture, data access is typically controlled by a centralized data team, which can create bottlenecks in the data pipeline and slow down data access. This can be particularly problematic in large organizations with many different data consumers with different needs and requirements.

In contrast, in a data mesh architecture, data access is decentralized and self-serve, which allows data consumers to access the data they need directly. This is achieved through data discovery, data catalogs, and data access APIs. With these capabilities, data consumers can search for and discover relevant data sets, understand their structure and quality, and access them using a standardized interface.

Current pain points

Without a data access layer in a data mesh architecture, there can be several pain points, including:

1. Data Inconsistency: Each data product team may use its own data storage, which can lead to inconsistent data across the organization. Without a centralized data access layer, it can be challenging to ensure that the data is up-to-date, accurate, and consistent.
2. Data Security: Data security is a critical concern in any organization. Without a data access layer, there may be limited control over who can access data, how data is accessed, and how it is protected. This can increase the risk of data breaches, leaks, and unauthorized access.
3. Data Governance: Data governance is essential for ensuring that data is properly managed, tracked, and audited. Without a centralized data access layer, it can be challenging to enforce data governance policies consistently across the organization.
4. Data Integration: Data mesh architecture emphasizes the need for data interoperability between different data products. Without a data access layer, integrating data from different sources can be a complex and time-consuming process.
5. Scalability: As the organization grows and the number of data products increases, managing and scaling without a centralized data access layer can become challenging. It can be difficult to ensure that all data products are performing optimally and meeting the organization's needs.
6. Complexity: Without a centralized data access layer, the complexity of managing data products and maintaining consistency can increase. This can result in higher costs, longer development cycles, and slower time-to-market.

Canner — The Data Access Layer of Data Mesh

Canner’s "Data Access Layer" is designed around four core principles:

1. Data Virtualization

The Data Access Layer is collaborative and distributed, with each silo or data source independently scalable or aggregated together.

2. Data Productization

Transform data models into domain-oriented datasets, which can be owned by data owners, shared, and governed by open APIs. This allows for interchangeable metadata and access rules, letting data speak the language of your business.

3. Data Authorization

Implement a consistent data authorization framework from data sources to data applications that is integrated with existing Identity and Access Management (IAM). This ensures that data authorization is consistent across data sources, IAM, and data applications.

4. Data Consumption

Data consumers can generate queries and APIs with intent and contextual settings. These are applied to the corresponding datasets via intent declaration, and then delivered to target consumers for final analytics and display.

By enabling self-serve data access, a data mesh architecture can improve the speed and efficiency of data consumption while also promoting data democratization and collaboration. This means that different teams and individuals across the organization can access and use data in a more agile and flexible way, without relying on a centralized data team to provision and manage data access. Furthermore, allowing data consumers to access data directly can promote a culture of data-driven decision-making, where individuals and teams are empowered to make decisions based on data rather than gut instincts or incomplete information.

"Our vision at Canner is to make all data access processes possible on a single platform," said the company's CEO Howard. "With Canner Enterprise v2, we are focused on improving the experience for data professionals and application users, enabling them to collaborate and interact with data products seamlessly."

Key features of Canner Enterprise v2 include a unified side panel for easy data exploration and access, as well as modular design that allows for different user personas to access different functionality based on their needs.

"We are thrilled to launch Canner Enterprise v2 and provide a more streamlined and user-friendly platform for data professionals and application users," said the CEO. "We believe that this new product will help organizations to better manage their data and collaborate more effectively, ultimately leading to improved decision-making and business outcomes."

For more information about Canner Enterprise v2, visit the company's website at https://cannerdata.com.

Data mesh is an architectural approach that emphasizes decentralization of data ownership and management, where each domain or product team is responsible for managing its own data. Data mesh also emphasizes the need for data interoperability and standardization between different domains, achieved through data contracts and APIs.

On the other hand, data fabric is an architectural approach that aims to provide a unified view of data across the organization, regardless of where the data is stored or managed. A data fabric typically uses a data virtualization layer to provide a single access point to all data sources, making integrating data across different systems easier.

Differences between the two architectures

1. Ownership and management of data: In a data mesh architecture, each domain or product team manages its own data. For example, a team responsible for customer data would manage its own customer database, and a team responsible for product data would manage its own product database. On the other hand, in a data fabric architecture, data is centrally managed and governed, and the data fabric provides a unified view of all data across the organization, regardless of where the data is stored.
2. Data interoperability and standardization: In a data mesh architecture, data interoperability and standardization are achieved through the use of data contracts and APIs, which define how data should be exchanged and accessed between different domains. Each domain or product team is responsible for defining its own data contracts and APIs. In contrast, a data fabric architecture typically uses a data virtualization layer to provide a unified view of all data across the organization, making integrating data from different sources easier.
3. Data governance and security: In a data mesh architecture, data governance and security are decentralized, with each domain or product team responsible for managing its own data security and governance policies. Data governance and security are centrally managed and enforced across the organization in a data fabric architecture.
4. Scalability and flexibility: Data mesh architecture is designed to be flexible and scalable, making it easier to add new domains or product teams as the organization grows. Each domain or product team can manage its own data infrastructure, allowing them to choose the tools and technologies that best fit its needs. In contrast, a data fabric architecture is designed to provide a unified view of all data across the organization, making it easier to integrate and analyze data. However, this can be less flexible and more challenging to scale as the organization grows.

Implementing with Canner Enterprise

Both Data Mesh and Data Fabric are important architectural concepts for modern enterprises in data integration. Canner Enterprise designs to assist enterprises in implementing both Data Mesh and Data Fabric architecture.

Headless BI allows developers to embed analytics functionality into their own applications or workflows, without requiring users to switch to a separate BI tool or platform. This approach enables developers to build customized data-driven applications or integrate analytics capabilities into existing workflows, which can improve the user experience and productivity.

In a headless BI system, the data is typically stored in a separate data warehouse or data lake, and the analytics are performed using specialized tools such as SQL or Python. The results of the analytics can then be accessed programmatically using APIs, which can be integrated into other applications or services.

Headless BI can be particularly useful in modern, cloud-based environments where data and analytics are distributed across multiple systems and services. It can help organizations to build flexible, scalable analytics solutions that can be customized to meet their specific needs. However, it may require more technical expertise than traditional BI systems, as developers need to understand how to access and manipulate data programmatically.

There are several benefits to using a headless BI approach:

1. Flexibility: With headless BI, developers can build custom analytics solutions that are tailored to their specific needs. They can choose the data sources, analytics tools, and APIs that best meet their requirements, and build the solution using the programming language and framework of their choice.
2. Scalability: Headless BI systems can be scaled up or down as needed, depending on the volume of data and the number of users. Developers can add or remove data sources, analytics tools, and APIs as the needs of the organization change.
3. Integration: Headless BI makes it easy to integrate analytics functionality into other applications and workflows. Developers can use APIs to access the analytics results programmatically and incorporate them into other systems or services.
4. Cost-effective: Headless BI can be a more cost-effective solution than traditional BI tools, especially for organizations with complex or unique data requirements. Instead of paying for a full suite of BI tools, organizations can build custom solutions that meet their specific needs.
5. Governance: With headless BI, organizations can maintain centralized control over their data, even as it is used in different applications and services. This can help ensure that data is consistent, accurate, and up-to-date, and can also help with compliance and regulatory requirements.

Key features of headless BI:

1. Decoupled architecture: In a headless BI system, the data analytics and visualization layers are decoupled. This means that the data and analytics can be accessed programmatically, without the need for a dedicated user interface.
2. API-driven: Headless BI relies on APIs (Application Programming Interfaces) to make the data and analytics available to other applications and services. APIs can be used to access data, perform analytics, and retrieve results.
3. Customizable: Headless BI systems are highly customizable. Developers can choose the analytics tools and data sources that best meet their needs and build custom solutions that are tailored to their specific requirements.
4. Developer-focused: Headless BI is focused on providing analytics capabilities to developers, rather than end-users. This means that it may require more technical expertise to implement and use, but it also provides greater flexibility and control over the analytics process.
5. Governance: Headless BI systems provide centralized control over data, even as it is used in different applications and services. This can help ensure that data is accurate, consistent, and up-to-date, and can also help with compliance and regulatory requirements.

Canner Enterprise to achieve Headless BI Architecture

By using Canner Enterprise, companies can achieve a true Headless BI architecture. This means that the BI layer is separated from the data layer, allowing companies to easily connect to any data source, regardless of where the data resides. This enables companies to achieve a more agile and flexible approach to BI, enabling them to quickly and easily adapt to changing business needs.

With Canner Enterprise, companies can easily connect to any data source, including both traditional and modern sources such as MS SQL, Databricks, SAP, Oracle, and Big Data platforms. Canner Enterprise's powerful data virtualization technology allows companies to combine and transform data from multiple sources, enabling them to create a single, unified view of their data.

Canner Enterprise also offers robust security features, allowing companies to authorize different groups and users to access specific data sources and BI content. This ensures that data is protected and that only authorized users have access to sensitive data.

Additionally, Canner Enterprise offers powerful optimization features that allow companies to optimize their BI architecture based on different use cases and scenarios. This enables companies to achieve advanced analytics and custom BI, allowing them to gain deeper insights into their data and make more informed business decisions.

1. Batch vs. Real-time Processing:

Lambda Architecture involves processing data in both batch and real-time mode. In contrast, Kappa Architecture only processes data in real time.

2. Complexity:

Lambda Architecture is more complex than Kappa Architecture as it involves building and maintaining two separate processing pipelines: real-time data processing and batch processing. In contrast, Kappa Architecture has only one processing pipeline.

3. Data Storage:

In Lambda Architecture, data is stored in two different storage systems: one for real-time processing and another for batch processing. Kappa Architecture, on the other hand, only requires one storage system, which can store both real-time and batch data.

4. Fault Tolerance:

Lambda Architecture is more fault-tolerant than Kappa Architecture because it has redundant processing pipelines, one for real-time data processing and another for batch processing. In case of any failure, the system can switch to the other pipeline. In contrast, Kappa Architecture has only one processing pipeline, which can result in data loss in case of any failure.

5. Scalability:

Both architectures are highly scalable and can handle large volumes of data. However, Lambda Architecture is better suited for scenarios where there is a high volume of data and the processing requirements are complex.

Which is better?

Lambda Architecture and Kappa Architecture are both useful for different scenarios and have their own advantages and disadvantages. The choice between the two architectures depends on the specific needs of the system being designed. Here are some factors that can help determine which architecture is better:

1. Real-time Processing: If the system requires real-time processing of data and the data processing requirements are not too complex, Kappa Architecture can be a better choice as it has a simpler design and allows for faster processing of data.
2. Complex Data Processing: If the system requires complex data processing, involving both batch and real-time processing, Lambda Architecture can be a better choice as it allows for data processing in both modes and can handle complex data processing requirements.
3. Fault Tolerance: If the system requires high fault tolerance and redundancy, Lambda Architecture can be a better choice as it has redundant processing pipelines, which can help in case of any failures.
4. Data Storage: If the system requires efficient and cost-effective data storage, Kappa Architecture can be a better choice as it only requires one storage system for both real-time and batch data.
5. Flexibility: If the system requires flexibility to handle different types of data sources and processing requirements, Lambda Architecture can be a better choice as it can handle both structured and unstructured data and provides the flexibility to handle complex processing requirements.

Achieve Lambda Architecture with Canner

Fortunately, Canner Enterprise provides a software solution that makes it easy to achieve Lambda Architecture. With Canner Enterprise, you can quickly build and deploy Lambda Architecture-based data processing systems, enabling you to process large volumes of data in a fault-tolerant and scalable way. Whether you're dealing with big data, real-time data processing, or complex data processing requirements, Canner Enterprise can help you achieve Lambda Architecture without the hassle of building and maintaining the system from scratch.

As we begin this dynamic year, we are preparing for new challenges and opportunities. At the Kick-off Meeting, we reflect on the past year's accomplishments, discuss future plans and strategies, and strengthen teamwork and culture. This year's theme is "I am a Canner, I am Possible." This theme expresses our confidence in ourselves, our team and our company. When we combine our ability, belief and determination, we can create the impossible!

In this year's Kick-Off Meeting we are divided into two parts:

In the first half of the activity, each department shared the achievements of the past year and the goals and plans of this year with the colleagues of the whole company. It not only allows everyone to understand the operation and development of each team from the entire company level, Sales Team, Marketing Team, Dev Team, Support Team and Operation Team, but also allows company colleagues to better understand the cooperation and contributions between various departments.

In the second half of the event, we also invite colleagues from the R&D team to share some amazing R&D technologies and how they respond to market needs. These technologies and thought processes not only have an important impact on product development, but also reflect our professionalism and innovation in the field of technology. Through such exchanges and sharing, colleagues throughout the company have a better understanding of our products and feel our enthusiasm and commitment to products and technologies.

This time we chose Taoyuan as the event location. This is not only to allow team members to leave the office environment, but also to inspire everyone's creativity in this way, and encourage everyone to adopt the "think outside of the box" way of thinking , concentrate on sharing and communicating in the kick-off meeting. In such an environment, every colleague can feel a new atmosphere and get more inspiration from it. In addition, we also set an interesting dress code theme on the day of the event: "Canner". The activity is carried out in the form of a team competition. Each team member can stimulate each other's creativity, observation and association through team discussion, and can also promote the interaction and understanding between team members.

Team Vulcan!

I am Canner, I am possible!

Data Mesh Live!

Four clouds~

The ASK ME trio (see the lower right corner of the official website), which won a lot of prizes, not only looks alike, but also has interactive services, which is super exciting!

Canner is a team of trust, commitment and dedication. Everyone puts their commitment into practice and is fully committed to achieving the company's goals and vision. At the same time, we also encourage team members to communicate effectively in an independent thinking manner. When we own our ideas and communicate them clearly, we can work together better to create great products and deliver great services. At the same time, we also pay attention to the balance between quality and speed. In our work, we plan our decisions on a user-oriented basis. Our goal is to maximize speed and efficiency while maintaining product and service quality. Moreover, we uphold the philosophy of "starting with the end in mind", which helps us better manage our time and goals, and better manage our plans and strategies.

In the end, teamwork is the key to success. At Canner, we are united and work together to overcome difficulties and challenges. I am a Canner, I am Possible!

The first session was given by Clkao, who shared with us a lot from his in-depth technical insights and practical application cases. Clkao's sharing discussed the cutting-edge development and future trends of LLM, giving participants a deeper understanding of the application and advantages of language models.

For the second session, we invited Cooper, the project leader of Accio. He introduced us to the exciting project of Accio, and joined Ming-Han on the difficulties encountered in the process of integrating with ChatGPT. This sharing is very valuable for people who are exploring similar integrations. The experience sharing of Cooper and Ming-Han saved us a lot of detours.

SELECT * FROM DataMeetup is not only a platform for learning and sharing, but also an opportunity for industry exchanges and cooperation. We have seen scenes where professionals from different backgrounds communicate with each other.

This kind of cooperation and cohesion will undoubtedly bring more innovations and breakthroughs to the AI ​​and open source communities. Thanks to Clkao, Cooper and Ming-Han for their great sharing. Thanks to all the partners who participated in this SELECT * FROM Data Gathering, your enthusiastic participation made this event even more special! We look forward to hosting more events like this in the future and continue to promote the thriving development of the AI ​​and open source community!

Originally post on: https://vulcansql.com/blog/powering-rapid-data-apps-with-vulcansql

What Are the Challenges ?

When it comes to building data applications on top of your data warehouse, several obstacles can get in the way. Here are the primary obstacles you might come across:

1. High Query Latency: Customer-facing applications require speedy responses, often within milliseconds. Traditional data warehouses, however, are optimized for analytical workloads, which might lead to slow query responses that could negatively impact your application's user experience.

2. Security Concerns: In a world where data breaches are all too common, implementing an application-specific security layer is vital. This is especially important for multi-tenant environments where each user must access only their own data.

3. Cost Considerations: The scalability of your applications to serve a large number of concurrent users can bring about cost challenges. As the user base grows, the associated cost of managing a data warehouse might skyrocket. Striking a balance between scalability and cost can be quite a struggle.

Meet VulcanSQL

VulcanSQL, a data API framework built specifically for data applications, empowers data professionals to generate and distribute data APIs quickly and effortlessly. It takes your SQL templates and transforms them into data APIs, with no backend expertise necessary.

One of the defining attributes of VulcanSQL is its strategic use of DuckDB's exceptional capabilities as a caching layer. This synergistic pairing permits VulcanSQL to deliver low-latency APIs, presenting an optimal solution for scenarios where traditional data warehouses may fall short.

By utilizing VulcanSQL, you can move remote data computing in cloud data warehouses, such as Snowflake and BigQuery to the edge. This embedded approach ensures that your analytics and automation processes can be executed efficiently and seamlessly, even in resource-constrained environments.

Visualize a typical use case, where you, a data engineer or analytics engineer, regard the historical data in your data warehouse as "cold data" and store the data frequently accessed or relevant to the application within VulcanSQL as "hot data".

Now, let's see how straightforward it is to write a VulcanSQL API.

Crafting a Data API

Let's assume we are creating an API that will provide the daily revenue for the past three years.

For instance, I will set up a VulcanSQL server and use BigQuery as the data warehouse with TPC-H SF1 data within. We need to handle two things first, but I won't delve into the specifics here.

• Project configuration: Establish project-related configurations. For more information, visit: https://vulcansql.com/docs/api-plugin/overview
• Connecting to a data source: For more details, visit https://vulcansql.com/docs/connectors/overview. We support BigQuery, Snowflake, PostgreSQL, and Clickhouse.

Now let's dive into the most critical aspect - writing our APIs.

Step 1. Crafting APIs with SQL Templates

After initializing your VulcanSQL project, we can begin constructing your API using SQL templates. Let's develop a daily_revenue.sql file, which contains the main business logic of the API, all written in SQL.

As an example, I'll create an API that responds with the daily revenue for a specific period.

{% cache %}
select
  *
from daily_revenue
  where orderdate >= {{ context.params.startdate }}
  and orderdate <= {{ context.params.enddate }}
{% endcache %}

• Fetching from Cache: The {% cache %} ... {% endcache %} directive fetches data from the DuckDB caching layer.
• Dynamic Parameter: context.params.xxx stands for the values acquired from the query parameters. VulcanSQL uses your template to furnish users with data they specify via parameters.

Step 2. Configuring your API

Next, let's develop a daily_revenue.yaml file to configure your API.

urlPath: /daily_revenue
cache:
- cacheTableName: daily_revenue
  sql: "select sum(totalprice), orderdate from cannerflow-286003.tpch_sf1.orders where orderdate >= '1996-01-01' group by orderdate "
  profile: bq
  refreshTime:
      every: "1d"
profile: bq

Here we've defined an API with the path /daily_revenue. In the cache settings, we specific following configurations:

• cacheTableName: The name of the table in DuckDB that will store the cached data.
• sql: The SQL query that will be executed to extract data from BigQuery.
• refreshTime: The frequency at which the cache will be refreshed. In this case, it's daily.
• profile: We created a BigQuery profile in the project configuration step. Here, we specify the profile to use. Check out the Connecting to Data Sources for more information.

Once you start the VulcanSQL server, it syncs data from BigQuery to DuckDB for caching and then serves it at the /daily_revenue endpoint.

Performance Evaluation

In this scenario, we've extracted aggregated data from the data warehouse and transformed a SQL template into Data APIs using VulcanSQL. Now, let's assess the performance of VulcanSQL.

To evaluate if VulcanSQL could serve as a low-latency data API, especially in handling multiple concurrent requests, I conducted a series of load tests using k6.io.

Here's a look at a basic request. The load test was run for a span of 30 seconds. For a concurrency level of 1 and 10, VulcanSQL managed to handle the load within an impressive 100 milliseconds!

# concurrent 1, ran for 30 sec
med 9ms
p(95) 13ms

# concurrent 10, ran for 30 sec
med 48ms
p(95) 71ms

Even at a concurrency level of 100, VulcanSQL kept the response times within 500ms.

# concurrent 50, ran for 30 sec
med 223ms
p(95) 267ms

# concurrent 100, ran for 30 sec
med 422ms
p(95) 492ms

And with 300 concurrent users, the response time managed to serve within approximately a second, still showing impressive scalability.

# concurrent 300, ran for 30 sec
med 1.0s
p(95) 1.3s

All our source code is available on Github. Feel free to check it out!

Summary

In today's data-driven world, businesses rely heavily on their data warehouses. These rich repositories serve as the backbone for collecting and processing tons of data, turning raw numbers into valuable insights.

However, as more and more organizations look to build data applications atop these data warehouses, a new need arises - the need for a robust data API framework. This framework should be designed to overcome traditional obstacles like high query latency, security concerns, and cost issues, enabling organizations to fully leverage their data and drive their operations to new heights.

Are you ready to step up your data application game with VulcanSQL? Check out our quickstart guide here and embark on a journey towards more efficient, secure, and cost-effective data applications.

Finally, if VulcanSQL resonates with you, please consider starring us on GitHub!

The K-Startup Grand Challenge(KSGC) is not just a government program; it's a beacon of hope and opportunity for global startups aspiring to delve into the Korean ecosystem. Whether the goal is to raise a new round of investment funds or expand business operations, KSGC stands as a gateway to endless possibilities. Being chosen as one of sixty teams from a massive pool of startups was not just an honor; it was a validation of our vision and hard work.

As we approached the Demo Day, held from October 31st to November 2nd, our objectives were clear. This was our platform to impress a discerning panel of judges, consisting of investors and venture capitalists, who would evaluate our presentations and decide which teams could bring more business, job creation, and innovation to Korea.

The Genesis of Our Adventure

The journey to KSGC began with an intense selection process. Being chosen from over 6,000 startups was exhilarating and overwhelming. We spent countless hours preparing before arriving in Korea, fueled by ambition and a desire to make a mark.

Our aspirations were high. We aimed not only to showcase our startup but also to immerse ourselves in an environment that was a melting pot of innovation and cultural exchange.

South Korea - A New Chapter:

Landing in Korea was like stepping into a new world. The initial days were a whirlwind of excitement, learning, and cultural adaptation. The support system provided by KSGC was commendable, offering networking opportunities that were invaluable.

Every day was a new learning experience, from understanding the nuances of the Korean market to interacting with fellow entrepreneurs from around the globe. These moments were not just memorable; they were transformative.

The Hustle – Gearing Up for Demo Day:

Preparing for Demo Day was a journey in itself. It involved rigorous planning, relentless practice, and fine-tuning our pitch. The mentorship and guidance we received were unparalleled, helping us evolve our presentation and strategy.

Our team faced challenges, but our collective spirit and motivation never waned. We supported each other through late nights and early mornings, always focused on the goal ahead.

Demo Day – The Pinnacle of Our Journey:

The atmosphere on Demo Day was electric. The anticipation and energy were palpable as each team prepared to showcase their months of hard work. From the Opening Ceremony to the individual pitching sessions, every moment was charged with potential and possibility.

Our presentation was a culmination of our journey, a story we were proud to tell. We learned about expanding in a foreign ecosystem and received invitations from a few accelerators who were in the audience, seeing great potential in our product for the Korean market.

Conclusion:

As the Demo Day curtains drew to a close, we carried with us more than just the exhilarating experience and the invaluable feedback; we held a renewed sense of purpose and a blueprint for the future. This journey has been a crucible of growth, and we emerge from it not just as contenders, but as a formidable force ready to make our mark on the world. Stay tuned as we turn these lessons into stepping stones towards success.

It’s the end of 2023! I'm so excited and overwhelmed by so many blog posts talking about and mentioning “Semantic Layer” recently… it's definitely a fantastic and big year in 2023 for “Semantic Layer”.

The Definite Future

Earlier in 2023, there was a big news about dbt's acquisition Since then, the data ecosystem has been eagerly waiting for dbt's next move into the semantic layer. In 2023 October, during the dbt Coalesce, dbt announced its next-generation "dbt semantic layer," which envisions the future of how the semantic layer looks like. Recently, Jen Grant, the COO of Cube, dubbed 2023 as the year of the semantic layer in a post. She stated that "While Cube and the semantic layer have been around for a long time, it was only in 2023 when the stand-alone semantic layer category turned from a cool idea to a necessary ingredient in any data stack."

Let’s look at macro statistics, Google Trend also shows starting from 2022, the term “Semantic Layer” started to gain global attention. In 2023, it is at its peak. 2024 will be more exciting for “Semantic Layer”!

Not only US, but also Asia

Many US-based companies shared customer’s enthusiasm around “Semantic Layer”.

At Canner, we also see the same trend happening not only in the US market but also in Asia. This year, we are fortunate to provide and implement our solution in banks, insurance companies, manufacturing, gaming, and retail sectors in the Asia region. By providing Canner Enterprise to our clients we still see “Semantic Layer” is still in an early stage, and have so much great potential and possibility in the upcoming years.

As many new initiates and innovations are still happening in Canner. Today I would like to share our vision and focus of “Semantic Layer” towards 2024.

Our 5 Key focus of innovating the Universal Semantic Layer in 2024

1. Query Virtualization Compatibility

The semantic layer operates as the same fundamental logic across all data applications, including BI, AI, and advanced analytics tools. The way to achieve this is that Semantic Layer solutions will implement technology like Query Virtualization, which is an essential part of a semantic layer; through virtualization, enterprise data consumers can access data in a vendor-agnostic way; behind the scenes, the semantic layer will automatically rewrite queries and pushdown into source-specific SQL language with optimization and automation.

As the semantic layer grows into mainstream adoption, the compatibility problem for traditional and latest sources and data applications must be gracefully solved. At Canner, I've supported many sources and data applications that can connect to the Semantic Layer, sources like Informix, Sybase, Oracle, BigQuery, and applications such as SSIS, SSAS, SAS, Tableau, Power BI, Metabase, etc.

In 2024, we continue to expand the compatibility around data applications and sources.

2. Context-aware Metrics Access

The future of metrics, we believe is dynamic and composible, which means that the metrics should be aware of the context. This will allow for the automatic composition of metrics when different users or queries are involved without the need for creating views and tables. To achieve composability, a metrics interface for SQL and APIs is required.

We will need a metrics interface that is a well-defined shared boundary between metrics producers and data consumers. Semantic modeling is defined using the Model Definition Language (MDL), which allows the data team to expose a consistent, strongly typed, and extendable interface. The client-driven architecture retrieves data tailored to users' needs without knowing any data structure in the metrics. It prunes and filters the correct data out, gratefully decoupling the computation logic and data serving. This enforces data consistency and fast performance by reducing huge chunks of over-fetching data in application tooling.

3. Governance in Business Collaboration

Governance is crucial for any data-related tools used by enterprise customers. The Semantic Layer plays a unique role in the modern enterprise data stack by bridging the gap between data and business teams. This means that many complex governance issues can only be resolved through collaboration between these two teams.

In the future, the governance of the Semantic Layer will be fully integrated with workflow processes. This will include data-sharing processes, approval workflows between departments, and cross-functional teams. The integration will be built on top of basic functions such as auditing trails, granular data access control, lineage, notification, and data policies that will also be incorporated into the Semantic Layer. As a result, teams and departments can collaborate seamlessly without switching between different platforms.

4. Enable AI in Data Analytics

According to recent studies, the Semantic Layer can help prevent AI hallucinations. To achieve this, a semantic layer must be established to provide adequate context and semantics to data, as well as define standard data and metric definitions across sources. This will help ensure that the necessary information is provided.

Direct access between AI and raw data can lead to security vulnerabilities. To mitigate this, generating SQL through the semantic layer can ensure granular access control policies are in place.

5. Open-source Semantic Layer

We believe that openness is crucial for the ecosystem, and some people in the community are calling for the semantic layer to have open standards. We plan to open-source our core - the Open Semantic Layer. Our goal is to create query standardization across BI, data science, AI, and advanced analysis tools through open definition standards and query engine. This will help enterprises solve the problem of inconsistency that arises from multiple data definitions in different application tools.

Our plan includes three main design components: Semantic Modeling, Semantic Definition Language, and Standard Protocol. Technical engineers can define the Semantic Layer via a syntax similar to GraphQL to define metric definitions. In addition, we provide a self-service UI interface for non-technical personnel to establish semantic standards across different application tools. We will also provide a standard SQL query via the PostgreSQL wire protocol standard, which most data applications can query against natively and API interface accessible by any application tool, ensuring that all applications can be managed and applied under a unified data governance framework.

Hello! 2024!

It's exciting to get started in 2024 and beyond! We will share more details in the upcoming blog posts and sharings, see you there!