Project MyCuan(PPOB) RestAPI

Published by Ardhi Ramadhani on June 13, 2022

Introduction

MyCuan is an application for bill payment services and online transactions available in real-time 24/7, known for its speed and security. MyCuan provides several product categories, including Top-up, bills, entertainment, and digital wallets.

Note: This post focuses on the REST API design & implementation.
Configuration and schema deployment design for the REST API was implemented by me.

Link to the repository: backend-mycuan repository

Technical Implementation

Technical Stack

We chose the following technologies for our implementation:

• Golang: For its performance and concurrency support
• Echo Framework: A high performance, minimalist Go web framework
• GORM: The fantastic ORM library for Golang, simplifying database operations
• PostgreSQL: Our primary database for its reliability and feature set
• JWT: For secure authentication
• Docker: For containerization and easy deployment
• AWS S3 and RDS: For object storage and managed database services

Database Design

We started with a comprehensive Entity-Relationship Diagram (ERD) to map out our data structure. Key entities in our schema include:

• Users
• Transactions
• Products
• Providers
• Categories
• Wallets
• We ensured proper relationships between these entities using foreign keys, allowing for efficient data retrieval and maintenance.

GORM played a crucial role in our database interactions. It allowed us to:

• Define models as Go structs
• Automatically create database tables from these structs
• Perform CRUD operations with minimal boilerplate code
• Handle relationships between entities efficiently

API Design

Our API design followed RESTful principles, with clear separation between user and admin endpoints. We used appropriate HTTP methods (GET, POST, PUT, DELETE) for different operations. Some key endpoint groups include:

• Authentication (login, logout, registration)
• User operations (profile management, transactions, wallet)
• Admin operations (user management, product management, transaction oversight)
• We also implemented versioning to ensure backward compatibility as the API evolves.

API Documentation with Swagger

To ensure clear communication and ease of integration for frontend developers and other stakeholders, we created comprehensive API documentation using Swagger. Our API documentation is publicly accessible at:

https://app.swaggerhub.com/apis/ppob-mycuan/api/1.0.2#/servers

This Swagger documentation provides:

• Detailed information about all available endpoints
• Request and response schemas for each endpoint
• Authentication requirements
• Example requests and responses
• API version information (currently at version 1.0.2) By maintaining up-to-date Swagger documentation, we've significantly improved the developer experience for those integrating with our API.

Development Workflow

We used Git for version control, following a branching strategy that allowed for feature development, bug fixes, and easy integration. Our workflow included regular commits and pull requests to maintain code quality.

As part of our commitment to maintaining high-quality documentation, we ensured that any changes to the API were immediately reflected in our Swagger documentation.

Code Structure

One of the key aspects of our MyCuan PPOB REST API project is its well-organized structure. We've adopted a clean architecture approach, which has significantly contributed to the maintainability and scalability of our codebase. Let's take a closer look at how we've structured our project:

1. This structure is designed to separate concerns and make our codebase more modular and easier to maintain. Let's break down the key components:
2. Root Directory: Contains configuration files, Docker setup, and the main entry point of our application.
3. Businesses Directory: This is where the heart of our application resides. It contains all the core business logic, separated by domain (e.g., users, transactions, wallets). Each subdirectory represents a distinct feature of our PPOB system.
4. Controllers Directory: Handles all the HTTP-related operations. It's responsible for processing requests, calling the appropriate business logic, and formatting responses. The separation of request and response handling (as seen in the products subdirectory) allows for clear input validation and output formatting.
5. Drivers Directory: Manages our database interactions. We're using PostgreSQL, and this directory contains all the database-specific code. Each subdirectory corresponds to a domain and handles the respective database operations.
6. Utils Directory: Contains utility functions that are used across the project, promoting code reuse and maintaining DRY (Don't Repeat Yourself) principles.

This structure allows us to maintain a clear separation of concerns:

• The business logic is isolated in the businesses directory, making it easy to modify core functionality without affecting other parts of the system.
• HTTP handling is confined to the controllers directory, allowing us to change our API structure or even the web framework without touching the business logic.
• Database operations are encapsulated in the drivers directory, making it possible to switch databases or modify data access patterns with minimal impact on the rest of the codebase.

By adopting this clean architecture, we've created a system that's not only robust and efficient but also highly maintainable and adaptable to future changes. This structure has been instrumental in allowing our team to work on different parts of the system simultaneously, reducing conflicts and making our development process smoother and more efficient.

Configuration and Infrastructure Setup

In our MyCuan PPOB REST API, we've implemented robust configuration management and infrastructure setup to ensure smooth operation and scalability. Let's dive into some key aspects of our setup:

Environment Configuration

We use environment variables to manage our configuration securely. Here's a snippet of our .env file structure:

DB_USERNAME=""
DB_PASSWORD=""
DB_HOST=""
DB_PORT=""
DB_NAME=""
JWT_SECRET_KEY=""
DB_TEST_NAME=""
PORT=""
AWS_S3_REGION=""
AWS_S3_BUCKET_KEY_ID=""
AWS_S3_BUCKET_SECRET_KEY=""
AWS_S3_BUCKET_NAME=""

This approach allows us to easily manage different configurations for development, testing, and production environments without hardcoding sensitive information.

GORM Integration

GORM has been instrumental in simplifying our database operations. We use it across our drivers/postgresql directory to interact with our PostgreSQL database. Here's a brief example of how we've set up a model:

AWS S3 Integration

We've implemented file upload functionality using AWS S3. Here's a look at our UploadToS3 function:

func UploadToS3(c echo.Context, folder string, filename string, src multipart.File) (string, error) {
	SECRET_KEY := util.GetEnv("AWS_S3_BUCKET_SECRET_KEY")
	KEY_ID := util.GetEnv("AWS_S3_BUCKET_KEY_ID")
	REGION := util.GetEnv("AWS_S3_REGION")
	BUCKET_NAME := util.GetEnv("AWS_S3_BUCKET_NAME")
 
	configS3 := &aws.Config{
		Region:      aws.String(REGION),
		Credentials: credentials.NewStaticCredentials(KEY_ID, SECRET_KEY, ""),
	}
	s3Session := session.New(configS3)
	uploader := s3manager.NewUploader(s3Session)
	result, err := uploader.Upload(&s3manager.UploadInput{
		Bucket: aws.String(BUCKET_NAME),
		Key:    aws.String(folder + filename),
		Body:   src,
	})
	if err != nil {
		log.Fatal(err)
		return "", err
	}
	return result.Location, nil
}

This function configures the AWS S3 client using environment variables, then uploads the file to the specified S3 bucket. It returns the URL of the uploaded file, allowing us to easily store and retrieve user-uploaded content.

Key features of our S3 integration:

Secure credential management using environment variables Flexible folder structure within the S3 bucket Error handling for upload failures

Database Configuration and Initialization

We've created a ConfigDB struct to manage our database configuration:

type ConfigDB struct {
    DB_USERNAME string
    DB_PASSWORD string
    DB_NAME     string
    DB_HOST     string
    DB_PORT     string
}

Our InitDB method uses this configuration to establish a connection to our PostgreSQL database:

func (config *ConfigDB) InitDB() *gorm.DB {
	connectionString := fmt.Sprintf("host=%s user=%s password=%s dbname=%s port=%s sslmode=disable TimeZone=Asia/Shanghai",
		config.DB_HOST,
		config.DB_USERNAME,
		config.DB_PASSWORD,
		config.DB_NAME,
		config.DB_PORT,
	)
 
	db, err := gorm.Open(postgres.Open(connectionString), &gorm.Config{})
	if err != nil {
		log.Fatalf("error when connecting to a database server: %s", err)
	}
 
	log.Println("connected to a database server")
	return db
}

This method constructs the connection string using the configuration parameters and establishes a connection using GORM. Key aspects of our database setup include:

• Flexible configuration allowing easy switching between different database instances
• Use of GORM for ORM functionality, simplifying database operations
• Error handling to catch and log any connection issues
• Our InitDB method uses this configuration to establish a connection to our PostgreSQL database:

Streamlined Database Migrations with GORM

In our MyCuan PPOB REST API project, we've implemented a straightforward and efficient approach to database migrations using GORM's AutoMigrate feature. This method allows us to quickly set up and update our database schema, which is particularly useful during the rapid development phase of our project. Let's take a closer look at how we've implemented this:

The DBMigrate Function

We've encapsulated our migration logic in a single function called DBMigrate. This function takes a GORM database instance as a parameter and uses it to automatically migrate our database schema based on our defined models. Here's the function:

func DBMigrate(db *gorm.DB) {
	db.AutoMigrate(
		&roles.Role{}, // role
		&users.User{}, // User
		&category.Category{},
		&wallets.Wallet{},
		&wallet_histories.WalletHistory{},
		&producttypes.ProductType{}, // ProductType
		&providers.Provider{},       // Provider
		&products.Product{},         // Product
		&transactions.Transaction{}, // Transaction
		&faq.FAQ{},
		// Provider
		// ProductType
		// ...
	)
}

This function leverages GORM's AutoMigrate method, which automatically creates or updates database tables based on the struct definitions of our models.

Benefits of This Approach

1. Simplicity: With just one function call, we can ensure that our database schema is up-to-date with our latest model definitions.
2. Rapid Development: During the early stages of development or when working on feature branches, this method allows us to quickly iterate on our data model without writing explicit migration files.
3. Consistency: It ensures that our database schema always matches our Go struct definitions, reducing the chance of inconsistencies between our code and database.
4. Comprehensive Coverage: As seen in the function, we're able to migrate all our models in one go, from roles and users to products and transactions.

Database Schema: A Comprehensive Look at Our ERD

A crucial aspect of our MyCuan PPOB REST API is its well-structured database design. The Entity-Relationship Diagram (ERD) provides a visual representation of our database schema, illustrating the relationships between different entities in our system. Let's dive into the details of our ERD:

Core Entities

1. Users: This table is the backbone of our user management system. It stores essential user information such as name, email, phone number, and password. Each user has a unique identifier (PK) and is associated with a role.
2. Roles: This table defines the different roles available in our system, allowing for role-based access control.
3. Wallet: Each user has an associated wallet, storing the user's balance. This table is crucial for managing user funds within our PPOB system.
4. Wallet_Histories: This table keeps track of all wallet transactions, providing a detailed history of cash inflows and outflows for each user's wallet.

Product-Related Entities

5. Product: Contains details about each product available in our PPOB system, including name, description, price, and stock information.
6. Provider: Represents the various service providers available in our system. Each product is associated with a provider.
7. Product_Type: Categorizes products into different types, allowing for easier product management and searching.
8. Category: Provides a higher-level categorization for product types, further organizing our product offerings.

Transaction Entity

9. Transactions: This is a central table in our system, recording all transactions made by users. It links to the user, product, and wallet, capturing essential details like transaction amount, date, and status.

Additional Features

10. FAQ: Stores frequently asked questions and their answers, supporting our customer service efforts.

Conclusion

The MyCuan(PPOB) RestAPI project showcases a robust, scalable, and well-structured backend solution for a comprehensive bill payment and online transaction system. By leveraging modern technologies like Golang, Echo Framework, and PostgreSQL, along with cloud services from AWS, we've created a high-performance API that can handle real-time transactions securely and efficiently.

Key takeaways from this project include:

1. The importance of clean architecture in creating maintainable and scalable code.
2. The value of comprehensive API documentation using tools like Swagger.
3. The benefits of using ORM tools like GORM for simplified database operations.
4. The significance of secure configuration management and infrastructure setup.
5. The power of well-designed database schemas in supporting complex business logic.

Link to the repository: backend-mycuan repository