Skip to content

Latest commit

 

History

History
198 lines (159 loc) · 5.31 KB

design-summary.md

File metadata and controls

198 lines (159 loc) · 5.31 KB
Raw

Connectivity Tester: Design and Implementation Summary

1. Overview and Core Functionality

1.1 Project Purpose

The Connectivity Tester is a Go-based CLI tool designed to test and manage network connectivity through various proxy servers. The project serves two main purposes:

  1. Testing and managing server connections
  2. Managing and testing SOAX client vantage points

1.2 Core Components

The project is structured into several key packages:

  • cmd: Contains the main CLI application
  • pkg/connectivity: Handles connection testing
  • pkg/database: Manages database operations
  • pkg/models: Defines data structures
  • pkg/server: Handles server-related operations
  • pkg/soax: Manages SOAX client operations
  • pkg/ipinfo: Handles IP information retrieval
  • pkg/fetch: Provides HTTP request functionality

1.3 Configuration Management

  • Uses Viper for configuration management
  • Configuration stored in YAML format
  • Supports multiple configuration paths
  • Manages sensitive credentials for database and SOAX services

2. Server Testing Implementation

2.1 Server Model

type Server struct {
    IP             string    `bun:",pk"`
    Port           string    `bun:",pk"`
    UserInfo       string    `bun:",pk"`
    FullAccessLink string    `bun:",unique,notnull"`
    Scheme         string    `bun:",notnull"`
    // ... other fields
}

2.2 Testing Process

  1. Initial Setup

    • Parse server access links
    • Extract IP, port, and user information
    • Create database entries

  2. Connectivity Testing

    • Support for both TCP and UDP protocols
    • Concurrent testing implementation
    • Error handling and reporting

  3. Result Management

    • Database storage of test results
    • Error tracking (TCP and UDP)
    • Automatic removal of failed servers

3. SOAX Client Integration

3.1 Client Model

type SoaxClient struct {
    IP             string    `bun:",pk"`
    UUID           string    `bun:",unique,notnull"`
    SessionID      int64     `bun:",notnull"`
    SessionLength  int       `bun:",notnull"`
    // ... other fields
}

3.2 Implementation Process

  1. Client Acquisition

    • Random session ID generation
    • Dynamic URL construction
    • Unique IP verification

  2. Information Gathering

    • SOAX checker API integration
    • IPInfo.io integration for ASN information
    • Geographic and carrier information collection

  3. Data Management

    • Unique IP enforcement
    • Session tracking
    • Expiration time management

4. Database Design and Implementation

4.1 Schema Design

The database implementation uses the Bun ORM with PostgreSQL, featuring:

  • Composite primary keys for servers
  • IP-based primary keys for SOAX clients
  • Automatic timestamp management
  • Error tracking columns

4.2 Operations

  1. Server Operations

    • Upsert functionality
    • Bulk operations support
    • Concurrent access handling

  2. Client Operations

    • Unique IP enforcement
    • Session management
    • Partial result handling

5. CLI Implementation

5.1 Command Structure

connectivity-tester
├── add-servers [file]
├── test-servers [--tcp] [--udp]
└── get-clients [country] [type] [count]

5.2 Feature Implementation

  1. Server Management

    • File-based server addition
    • Selective testing capabilities
    • Error handling and reporting

  2. Client Management

    • Country-based client acquisition
    • Type-specific handling (residential/mobile)
    • Unique IP enforcement

5.3 Error Handling

  • Graceful failure management
  • Partial success handling
  • Detailed error reporting

6. Process Improvements and Iterations

6.1 Initial Version

  • Basic server testing
  • Single-threaded operations
  • Simple database operations

6.2 Major Improvements

  1. Concurrency

    • Implemented worker pools
    • Added concurrent testing
    • Added mutex-protected database operations

  2. Error Handling

    • Enhanced error reporting
    • Added partial success handling
    • Implemented automatic server removal

  3. Client Management

    • Added unique IP verification
    • Implemented attempt tracking
    • Added partial result saving

6.3 Performance Optimizations

  • Connection pooling
  • Concurrent testing
  • Efficient database operations

7. Future Considerations

7.1 Potential Improvements

  1. Scalability

    • Database sharding
    • Distributed testing
    • Load balancing

  2. Monitoring

    • Metrics collection
    • Performance tracking
    • Health checking

  3. Features

    • Additional testing protocols
    • Enhanced reporting
    • API integration

7.2 Maintenance Considerations

  1. Database

    • Regular cleanup of expired entries
    • Index optimization
    • Backup strategies

  2. Testing

    • Automated testing
    • Performance benchmarking
    • Error rate monitoring

8. Conclusion

The Connectivity Tester project has evolved into a robust tool for managing and testing network connectivity through various proxies. The implementation prioritizes:

  • Reliability through proper error handling
  • Performance through concurrent operations
  • Maintainability through modular design
  • Flexibility through configuration management

The iterative development process has allowed for continuous improvements and refinements, resulting in a tool that effectively handles both server testing and client management tasks.