feat: opensearch客户端初始化 feat: 索引模板接口 ai: 开发准则 chore: TDD流水线脚本

This commit is contained in:
mouseleee
2025-11-16 22:17:16 +08:00
commit da3883205c
18 changed files with 2707 additions and 0 deletions

14
.env.example Normal file
View File

@@ -0,0 +1,14 @@
# OpenSearch 配置示例
# 复制此文件为 .env 并填写实际值
# OpenSearch 集群端点
ES_ENDPOINT=
# AWS 区域
AWS_REGION=
# AWS 访问密钥 ID
AWS_ACCESS_KEY_ID=
# AWS 访问密钥密文
AWS_SECRET_ACCESS_KEY=

View File

@@ -0,0 +1,23 @@
---
applyTo: '**'
---
⚙️ AGENT 执行协议
本协议定义了 Agent 在任务执行期间必须遵守的操作指南和工作流程约束。严格遵守所有要点是不可谈判的。
一、执行前强制检查
强制审查开发准则:在启动任何开发或编码任务之前,你必须首先审查并确认遵守已经建立的开发准则(即系统 Prompt 或其他指令文件)。你所采取的任何行动都必须与这些总体规则保持一致。
二、设计与清晰度
清晰度优先:如果用户需求含糊不清、不完整,或者关键逻辑不确定,你必须暂停执行并向用户提出澄清问题,以确保完全理解后再进行后续实现。
零成本抽象原则KISS 原则):严格遵守“如无必要,勿增实体”的原则。不要引入任何对于所需功能并非严格必要的文件、模块、抽象或依赖。
三、开发工作流程TDD
测试驱动开发TDD你必须严格遵循测试驱动开发TDD流程。任何新功能或修复的实现工作只能在相应的、失败的测试用例已编写并保存之后才能开始。
四、输出与文件管理
输出限制:禁止使用说明文档:你不得生成任何详细说明如何使用或调用所实现代码的补充文档。只应生成源代码和必要的配置文件。
文件生成限制:对于任何给定的模块或逻辑单元,你至多只能生成一个主要的文档文件(例如,该模块的 README.md 或设计文件)。避免在多个文件之间冗余地生成文档。
测试编译是否成功的输出文件:只是为了验证是否可以编译成功,不需要保留这些输出文件。

View File

@@ -0,0 +1,975 @@
---
applyTo: '**/*.go'
---
# AI Agent 开发指令
本文档为 AI AgentGitHub Copilot提供开发本项目时的详细指令和指南。
## 项目背景
这是一个用于测试和实验 AWS OpenSearch Service 的 Go 项目。项目目标是创建生产级别的代码,并为未来转化为可复用的依赖库做准备。
## 核心原则
### 1. 代码质量要求(最高优先级)
**必须遵循 Go 官方最佳实践:**
- **Effective Go**: https://go.dev/doc/effective_go
- **Go Code Review Comments**: https://github.com/golang/go/wiki/CodeReviewComments
- **Go 标准库风格**: 参考标准库的代码风格和设计模式
**具体要求:**
```go
// ✅ 正确:导出的函数必须有文档注释,以函数名开头
// NewClient creates and initializes a new OpenSearch client with the provided configuration.
// It returns an error if the connection cannot be established.
func NewClient(cfg *Config) (*Client, error) {
// ...
}
// ❌ 错误:缺少注释或注释不以函数名开头
func NewClient(cfg *Config) (*Client, error) {
// ...
}
```
```go
// ✅ 正确:错误处理不能忽略
result, err := client.Search(ctx, query)
if err != nil {
return nil, fmt.Errorf("search failed: %w", err)
}
// ❌ 错误:忽略错误
result, _ := client.Search(ctx, query)
```
```go
// ✅ 正确:使用有意义的变量名
var maxRetryCount = 3
var connectionTimeout = 30 * time.Second
// ❌ 错误:使用无意义的缩写
var mrc = 3
var ct = 30 * time.Second
```
### 2. 命名规范
**包名:**
- 小写、简短、单数形式
- 避免使用下划线或驼峰命名
- 示例:`client``search``document`(不是 `opensearch_client`
**接口名:**
- 单方法接口以 `-er` 结尾:`Reader``Writer``Searcher`
- 多方法接口使用描述性名称:`IndexManager``DocumentService`
**变量和函数名:**
- 驼峰命名camelCase 或 PascalCase
- 缩写词保持一致大写:`URL``HTTP``ID``JSON`(不是 `Url``Http``Id``Json`
- 示例:`clientID``httpClient``parseURL`
**常量:**
```go
// ✅ 正确:使用驼峰命名
const DefaultTimeout = 30 * time.Second
const MaxRetries = 3
// 或者使用全大写(枚举类型)
const (
StatusPending = "PENDING"
StatusComplete = "COMPLETE"
)
```
### 3. 错误处理规范
**基本错误处理:**
```go
// ✅ 正确:包装错误,提供上下文信息
if err != nil {
return fmt.Errorf("failed to create index %q: %w", indexName, err)
}
// ✅ 正确:自定义错误类型(当需要时)
var ErrIndexNotFound = errors.New("index not found")
var ErrInvalidQuery = errors.New("invalid query syntax")
// ✅ 正确:错误检查使用 errors.Is 或 errors.As
if errors.Is(err, ErrIndexNotFound) {
// 处理索引不存在的情况
}
// ❌ 错误:吞掉错误信息
if err != nil {
return errors.New("something went wrong")
}
```
**Defer 中的错误处理:**
对于 `defer` 语句中的 Close() 等清理操作,**不能**完全忽略错误:
```go
// ✅ 正确:检查并记录 Close 错误
defer func() {
if closeErr := res.Body.Close(); closeErr != nil {
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
}
}()
// ❌ 错误:完全忽略错误(虽然能通过 linter但不是最佳实践
defer func() { _ = res.Body.Close() }()
// ❌ 错误不处理错误errcheck 警告)
defer res.Body.Close()
```
**原因说明:**
- HTTP Response Body 的 Close() 错误虽然通常不影响业务逻辑(数据已读取)
- 但完全忽略错误不符合 Go 最佳实践
- 检查并记录到 stderr 便于调试和监控
- 这种方式既满足 linter 要求,又提供诊断信息
```
### 4. 注释规范
```go
// ✅ 正确:包注释(在 package 语句之前)
// Package client provides a high-level interface for interacting with AWS OpenSearch Service.
// It handles connection management, authentication, and request signing.
package client
// ✅ 正确:导出类型的注释
// Client represents an OpenSearch client that manages connections and operations.
type Client struct {
// ...
}
// ✅ 正确:导出方法的注释
// Search executes a search query against the specified index.
// It returns the search results or an error if the operation fails.
func (c *Client) Search(ctx context.Context, indexName string, query Query) (*SearchResult, error) {
// ...
}
// ✅ 正确:复杂逻辑的内部注释
func processResults(data []byte) error {
// Parse JSON response
var resp Response
if err := json.Unmarshal(data, &resp); err != nil {
return err
}
// Validate response structure
if resp.Total == 0 {
return ErrNoResults
}
// Transform results
// ...
}
```
### 5. 代码结构规范
**文件组织:**
```go
// 1. 包声明和包注释
// Package client provides...
package client
// 2. 导入语句(分组:标准库、第三方库、本地包)
import (
"context"
"fmt"
"time"
"github.com/opensearch-project/opensearch-go/v2"
"es-demo/config"
"es-demo/models"
)
// 3. 常量
const (
DefaultTimeout = 30 * time.Second
)
// 4. 变量
var (
ErrInvalidConfig = errors.New("invalid configuration")
)
// 5. 类型定义
type Client struct {
// ...
}
// 6. 构造函数
func NewClient(cfg *Config) (*Client, error) {
// ...
}
// 7. 方法(接收者方法)
func (c *Client) Connect(ctx context.Context) error {
// ...
}
// 8. 其他函数
func validateConfig(cfg *Config) error {
// ...
}
```
**接口设计:**
```go
// ✅ 正确:接口定义在使用方,而非实现方
// 在 operations/search 包中定义
type DocumentStore interface {
GetDocument(ctx context.Context, id string) (*Document, error)
PutDocument(ctx context.Context, doc *Document) error
}
// ✅ 正确:保持接口小而专注
type Searcher interface {
Search(ctx context.Context, query Query) (*Result, error)
}
type Indexer interface {
CreateIndex(ctx context.Context, name string, settings map[string]any) error
DeleteIndex(ctx context.Context, name string) error
}
```
### 6. 测试规范
**文件命名:**
- 测试文件:`client_test.go`(对应 `client.go`
- 测试函数:`TestNewClient``TestClient_Search`
**测试结构:**
```go
func TestNewClient(t *testing.T) {
// 使用表驱动测试
tests := []struct {
name string
config *Config
want bool // 期望成功/失败
wantErr error
}{
{
name: "valid config",
config: &Config{
Endpoint: "https://example.com",
},
want: true,
wantErr: nil,
},
{
name: "nil config",
config: nil,
want: false,
wantErr: ErrInvalidConfig,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := NewClient(tt.config)
if !errors.Is(err, tt.wantErr) {
t.Errorf("NewClient() error = %v, wantErr %v", err, tt.wantErr)
return
}
if (got != nil) != tt.want {
t.Errorf("NewClient() got = %v, want %v", got != nil, tt.want)
}
})
}
}
// 集成测试(需要真实环境)
func TestClient_Search_Integration(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test in short mode")
}
// 测试逻辑...
}
```
**辅助函数:**
```go
// 测试辅助函数以小写字母开头(不导出)
func setupTestClient(t *testing.T) *Client {
t.Helper()
cfg := &Config{
Endpoint: "http://localhost:9200",
}
client, err := NewClient(cfg)
if err != nil {
t.Fatalf("failed to create test client: %v", err)
}
return client
}
func cleanupTestIndex(t *testing.T, client *Client, indexName string) {
t.Helper()
if err := client.DeleteIndex(context.Background(), indexName); err != nil {
t.Logf("cleanup failed: %v", err)
}
}
```
## 项目阶段考虑
### 当前阶段(实验期)
**允许的简化:**
1. **配置管理**:可以使用全局变量
```go
// config/config.go
package config
var (
// Global configuration (acceptable in experimental phase)
Endpoint string
AccessKey string
SecretKey string
Region string
)
func Init() {
// Simple initialization from environment variables
Endpoint = os.Getenv("ES_ENDPOINT")
AccessKey = os.Getenv("AWS_ACCESS_KEY_ID")
SecretKey = os.Getenv("AWS_SECRET_ACCESS_KEY")
Region = os.Getenv("AWS_REGION")
}
```
2. **日志系统**:基础的 log 包使用
```go
// utils/logger.go
package utils
import "log"
var (
// Global logger (acceptable in experimental phase)
Logger = log.New(os.Stdout, "[ES-Demo] ", log.LstdFlags)
)
func Info(msg string, args ...any) {
Logger.Printf("INFO: "+msg, args...)
}
func Error(msg string, args ...any) {
Logger.Printf("ERROR: "+msg, args...)
}
```
**但必须保持:**
- 代码的可读性和可维护性
- 清晰的职责划分
- 完整的错误处理
- 充分的注释
## 代码质量检查
### 使用 Linter 进行自动化检查
本项目使用 `golangci-lint` 进行代码质量检查。在生成或修改代码后,**必须**确保代码通过 linter 检查。
**配置文件:** `.golangci.yml`
**运行 Linter**
```bash
# 检查所有代码
golangci-lint run ./...
# 自动修复部分问题
golangci-lint run --fix ./...
```
**处理 Linter 输出:**
1. 认真阅读每一条 linter 警告和错误
2. 理解问题的根本原因
3. 按照 Go 最佳实践修复问题
4. 不要通过禁用 linter 规则来"修复"问题(除非有充分理由)
5. 如需禁用某个规则,必须在配置文件中注释说明原因
**常见 Linter 问题及解决方案:**
- `errcheck`: 错误未处理 → 添加错误处理逻辑
- `gofmt`: 格式不正确 → 运行 `go fmt ./...`
- `govet`: 可疑构造 → 重构代码
- `ineffassign`: 无效赋值 → 移除或修正赋值
- `staticcheck`: 静态分析问题 → 按建议修改
**集成到开发流程:**
- 在提交代码前,运行测试流水线(包含 linter 检查)
- 确保 linter 零警告、零错误
- CI/CD 流水线应包含 linter 检查步骤
## 常见模式和示例
### 客户端初始化模式
```go
// Config holds the configuration for the OpenSearch client.
type Config struct {
Endpoint string
Region string
AccessKey string
SecretKey string
Timeout time.Duration
}
// Client represents an OpenSearch client.
type Client struct {
config *Config
client *opensearch.Client
}
// NewClient creates a new OpenSearch client with the given configuration.
func NewClient(cfg *Config) (*Client, error) {
if cfg == nil {
return nil, fmt.Errorf("config cannot be nil")
}
if err := validateConfig(cfg); err != nil {
return nil, fmt.Errorf("invalid config: %w", err)
}
// Initialize OpenSearch client
osClient, err := opensearch.NewClient(opensearch.Config{
Addresses: []string{cfg.Endpoint},
// ... other settings
})
if err != nil {
return nil, fmt.Errorf("failed to create opensearch client: %w", err)
}
return &Client{
config: cfg,
client: osClient,
}, nil
}
func validateConfig(cfg *Config) error {
if cfg.Endpoint == "" {
return errors.New("endpoint is required")
}
if cfg.Region == "" {
return errors.New("region is required")
}
return nil
}
```
### 操作方法模式
```go
// SearchRequest represents a search query request.
type SearchRequest struct {
Index string
Query map[string]any
Size int
From int
}
// SearchResult represents the response from a search operation.
type SearchResult struct {
Hits []Document
Total int64
Took int64
}
// Search executes a search query against the specified index.
func (c *Client) Search(ctx context.Context, req *SearchRequest) (*SearchResult, error) {
if req == nil {
return nil, errors.New("search request cannot be nil")
}
if req.Index == "" {
return nil, errors.New("index name is required")
}
// Build and execute search
// ...
return result, nil
}
```
### Context 使用模式
```go
// 所有可能阻塞或需要超时控制的操作都应接受 context
func (c *Client) CreateIndex(ctx context.Context, name string, settings map[string]any) error {
// 使用 context 控制超时
req, err := http.NewRequestWithContext(ctx, "PUT", c.buildURL(name), body)
if err != nil {
return fmt.Errorf("failed to create request: %w", err)
}
// ...
}
// 调用示例
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
err := client.CreateIndex(ctx, "my-index", settings)
```
## 版本和依赖管理
**依赖选择原则:**
1. 优先使用最新稳定版本
2. 使用官方推荐的库(如 opensearch-go
3. 定期更新依赖(使用 `go get -u``go mod tidy`
**更新依赖时检查:**
```bash
# 查看可用更新
go list -u -m all
# 更新特定依赖
go get -u github.com/opensearch-project/opensearch-go/v2@latest
# 整理依赖
go mod tidy
```
## 测试流水线
### 流水线脚本要求
项目**必须**包含自动化测试流水线脚本,用于执行完整的代码质量检查流程。
**脚本命名规范:**
- Windows (PowerShell): `test.ps1`
- Linux/Mac (Bash): `test.sh`
**流水线阶段(按顺序执行):**
1. **Linting** - 代码质量检查
```bash
golangci-lint run ./...
```
2. **Build** - 编译验证
```bash
# 编译但不保留二进制文件
go build -o /dev/null . # Linux/Mac
go build -o NUL . # Windows
```
3. **Test** - 运行测试
```bash
go test -v -short -coverprofile=coverage.out ./...
```
4. **Cleanup** - 清理构建产物
```bash
# 删除编译输出、覆盖率文件等
rm -f *.exe *.test coverage.out coverage.html
```
### 流水线脚本检查
**在最终测试前AI Agent 必须检查:**
1. 当前平台是否存在对应的流水线脚本
2. 如果不存在但有其他平台的脚本,则根据当前平台创建对应版本
**平台检测和脚本生成逻辑:**
```go
// 伪代码示例
if platform == "Windows" {
if !exists("test.ps1") {
if exists("test.sh") {
// 将 test.sh 翻译为 test.ps1
createWindowsScript()
} else {
// 创建新的 Windows 脚本
createDefaultWindowsScript()
}
}
} else if platform == "Linux" || platform == "Mac" {
if !exists("test.sh") {
if exists("test.ps1") {
// 将 test.ps1 翻译为 test.sh
createUnixScript()
} else {
// 创建新的 Unix 脚本
createDefaultUnixScript()
}
}
}
```
### PowerShell 脚本示例 (test.ps1)
```powershell
#!/usr/bin/env pwsh
# 测试流水线脚本 - Windows PowerShell
param(
[switch]$SkipLint,
[switch]$SkipBuild,
[switch]$SkipTest,
[switch]$Verbose
)
$ErrorActionPreference = "Stop"
# 1. Linting
if (-not $SkipLint) {
Write-Host "=== Linting ===" -ForegroundColor Yellow
golangci-lint run ./...
if ($LASTEXITCODE -ne 0) { exit 1 }
}
# 2. Build
if (-not $SkipBuild) {
Write-Host "=== Build ===" -ForegroundColor Yellow
go build -o NUL .
if ($LASTEXITCODE -ne 0) { exit 1 }
}
# 3. Test
if (-not $SkipTest) {
Write-Host "=== Test ===" -ForegroundColor Yellow
go test -v -short -coverprofile=coverage.out ./...
if ($LASTEXITCODE -ne 0) { exit 1 }
go tool cover -func=coverage.out | Select-String "total:"
}
# 4. Cleanup
Write-Host "=== Cleanup ===" -ForegroundColor Yellow
Remove-Item -Path "*.exe","*.test","coverage.out","coverage.html" -ErrorAction SilentlyContinue
Write-Host "✅ Pipeline PASSED" -ForegroundColor Green
```
### Bash 脚本示例 (test.sh)
```bash
#!/usr/bin/env bash
# 测试流水线脚本 - Linux/Mac Bash
set -euo pipefail
SKIP_LINT=false
SKIP_BUILD=false
SKIP_TEST=false
VERBOSE=false
# 解析参数
while [[ $# -gt 0 ]]; do
case $1 in
--skip-lint) SKIP_LINT=true; shift ;;
--skip-build) SKIP_BUILD=true; shift ;;
--skip-test) SKIP_TEST=true; shift ;;
--verbose) VERBOSE=true; shift ;;
*) echo "Unknown option: $1"; exit 1 ;;
esac
done
# 1. Linting
if [ "$SKIP_LINT" = false ]; then
echo "=== Linting ==="
golangci-lint run ./...
fi
# 2. Build
if [ "$SKIP_BUILD" = false ]; then
echo "=== Build ==="
go build -o /dev/null .
fi
# 3. Test
if [ "$SKIP_TEST" = false ]; then
echo "=== Test ==="
go test -v -short -race -coverprofile=coverage.out ./...
go tool cover -func=coverage.out | grep total:
fi
# 4. Cleanup
echo "=== Cleanup ==="
rm -f *.exe *.test coverage.out coverage.html
echo "✅ Pipeline PASSED"
```
### 使用流水线
**日常开发:**
```bash
# Windows
powershell -ExecutionPolicy Bypass -File .\test.ps1
# Linux/Mac
chmod +x test.sh
./test.sh
```
**跳过特定阶段:**
```bash
# Windows - 跳过 linting
powershell -ExecutionPolicy Bypass -File .\test.ps1 -SkipLint
# Linux/Mac - 跳过 build
./test.sh --skip-build
```
**CI/CD 集成:**
```yaml
# GitHub Actions 示例
- name: Run test pipeline
run: |
if [ "$RUNNER_OS" == "Windows" ]; then
pwsh -File test.ps1
else
./test.sh
fi
```
### 流水线最佳实践
1. **构建产物管理:**
- 编译时不保留二进制文件(输出到 `/dev/null` 或 `NUL`
- 测试后自动清理所有临时文件
- 在 `.gitignore` 中排除所有构建产物
2. **失败快速原则:**
- 任何阶段失败立即终止流水线
- 返回非零退出码
- 提供清晰的错误信息
3. **可配置性:**
- 提供跳过特定阶段的选项
- 支持详细输出模式
- 允许自定义超时设置
4. **跨平台一致性:**
- 确保 Windows 和 Linux/Mac 脚本功能一致
- 使用等效的命令和选项
- 测试结果应该相同
## 文档要求
**代码文档:**
- 所有导出的标识符必须有文档注释
- 复杂的内部逻辑添加解释性注释
- 注释应解释"为什么"而不仅仅是"是什么"
**README 维护:**
- 添加新功能时更新功能列表
- 添加示例代码展示用法
- 更新配置说明
**示例代码:**
- 在 `examples/` 目录提供可运行的示例
- 示例代码必须可编译和运行
- 包含必要的错误处理
## AI Agent 工作流程TDD 原则)
当收到开发任务时,**必须严格遵循 TDD测试驱动开发流程**
### 1. Red红灯阶段- 编写失败的测试
**理解需求:**
- 明确要实现的功能和预期行为
- 确定输入、输出和边界条件
- 识别可能的错误场景
**设计 API**
- 定义函数签名和类型
- 设计清晰的接口
- 确定错误返回值
**编写测试(先于实现):**
```go
// 示例:先写测试
func TestNewClient(t *testing.T) {
tests := []struct {
name string
config *Config
wantErr error
}{
{
name: "valid config",
config: &Config{Endpoint: "https://example.com"},
wantErr: nil,
},
{
name: "nil config",
config: nil,
wantErr: ErrInvalidConfig,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := NewClient(tt.config)
if !errors.Is(err, tt.wantErr) {
t.Errorf("NewClient() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
```
**验证测试失败:**
- 运行测试,确认测试失败(因为功能还未实现)
- 确保测试失败的原因是正确的
### 2. Green绿灯阶段- 编写最小实现
**实现功能:**
- 编写最少的代码让测试通过
- 遵循本文档的所有规范
- 添加完整的文档注释
- 实现完整的错误处理
```go
// 示例:实现功能
func NewClient(cfg *Config) (*Client, error) {
if cfg == nil {
return nil, ErrInvalidConfig
}
// 最小实现
return &Client{config: cfg}, nil
}
```
**运行测试:**
- 确保所有测试通过
- 检查测试覆盖率
### 3. Refactor重构阶段- 优化代码
**代码重构:**
- 消除重复代码
- 改进命名和结构
- 优化性能(如需要)
- 确保代码可读性
**运行 Linter**
```bash
golangci-lint run ./...
```
**修复所有 Linter 问题**
**再次运行测试:**
- 确保重构后测试仍然通过
- 验证代码行为未改变
### 4. 验证和文档
**运行测试流水线:**
```bash
# Windows
powershell -ExecutionPolicy Bypass -File .\test.ps1
# Linux/Mac
./test.sh
```
**更新文档:**
- 更新模块 README如需要
- 更新项目 README如需要
- 添加使用示例(如需要)
### TDD 循环总结
```
┌─────────────────────────────────────┐
│ 1. 编写失败的测试 (Red) │
│ - 定义预期行为 │
│ - 验证测试确实失败 │
└──────────────┬──────────────────────┘
┌─────────────────────────────────────┐
│ 2. 编写最小实现 (Green) │
│ - 让测试通过 │
│ - 不过度设计 │
└──────────────┬──────────────────────┘
┌─────────────────────────────────────┐
│ 3. 重构优化 (Refactor) │
│ - 改进代码质量 │
│ - 运行 Linter │
│ - 确保测试仍然通过 │
└──────────────┬──────────────────────┘
┌─────────┐
│ 下一功能 │
└─────────┘
```
**重要提醒:**
- ❌ 不要先写实现再补测试
- ✅ 必须先写测试,然后写实现
- ✅ 每次只关注一个小功能
- ✅ 频繁运行测试,保持快速反馈
## 特殊指令
### 生成新模块时
1. 创建包目录和主文件
2. 创建对应的测试文件
3. 在 README 中更新项目结构
4. 在功能列表中标记进度
### 重构代码时
1. 确保向后兼容(除非明确要求破坏性变更)
2. 更新所有相关测试
3. 更新文档和注释
4. 运行完整测试套件
### 修复 Bug 时
1. 先编写复现 bug 的测试
2. 修复代码
3. 确保测试通过
4. 添加注释说明修复内容
## 禁止事项
**绝对不允许:**
- ❌ 忽略错误(使用 `_` 丢弃错误)
- ❌ 导出的标识符没有注释
- ❌ 使用 `panic` 处理常规错误(除非确实是不可恢复的错误)
- ❌ 在库代码中使用 `os.Exit`
- ❌ 硬编码配置值(应使用配置文件或环境变量)
- ❌ 使用全局可变状态(除了当前阶段允许的配置和日志)
**应当避免:**
- ⚠️ 过长的函数(超过 50 行考虑拆分)
- ⚠️ 过深的嵌套(超过 3 层考虑重构)
- ⚠️ 重复代码(提取为函数)
- ⚠️ 神秘的数字和字符串(使用命名常量)
## 参考资源
- [Effective Go](https://go.dev/doc/effective_go)
- [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments)
- [Go 标准库文档](https://pkg.go.dev/std)
- [OpenSearch Go Client](https://github.com/opensearch-project/opensearch-go)
- [AWS SDK for Go v2](https://aws.github.io/aws-sdk-go-v2/)
---
**总结**: 作为 AI Agent你的目标是生成生产级别的 Go 代码,遵循所有最佳实践,为将来转化为可复用库打下坚实基础。当有疑问时,参考 Go 标准库的实现方式。

37
.gitignore vendored Normal file
View File

@@ -0,0 +1,37 @@
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
# Test binary, built with `go test -c`
*.test
# Output of the go coverage tool
*.out
coverage
coverage.out
coverage.html
# Dependency directories
vendor/
# Go workspace file
go.work
# Environment variables
.env
.env.local
.env.test
# IDE specific files
.vscode/
.idea/
*.swp
*.swo
*~
# OS specific files
.DS_Store
Thumbs.db

67
.golangci.yml Normal file
View File

@@ -0,0 +1,67 @@
# golangci-lint 配置文件
# 详细文档: https://golangci-lint.run/usage/configuration/
run:
timeout: 5m
tests: true
modules-download-mode: readonly
output:
formats:
- format: colored-line-number
print-issued-lines: true
print-linter-name: true
sort-results: true
linters:
enable:
# 默认启用的 linters
- gosimple # 简化代码
- ineffassign # 检查无效赋值
- staticcheck # 静态检查
- unused # 检查未使用的代码
# 额外启用的 linters
- gofmt # 检查格式化
- goimports # 检查导入顺序
- misspell # 检查拼写错误
- gosec # 安全检查
- unconvert # 检查不必要的类型转换
- gocyclo # 检查代码复杂度
- goconst # 检查可以提取为常量的字符串
- whitespace # 检查空白符
linters-settings:
gocyclo:
min-complexity: 15
gosec:
excludes:
- G404 # 允许使用 math/rand
gofmt:
simplify: true
goimports:
local-prefixes: es-demo
issues:
exclude-dirs:
- vendor
exclude-use-default: false
max-issues-per-linter: 0
max-same-issues: 0
exclude-rules:
# 测试文件中允许一些宽松的规则
- path: _test\.go
linters:
- errcheck
- gosec
- gocyclo
- goconst
# 允许在 main 包中使用 fmt.Printf
- path: main\.go
linters:
- forbidigo

256
README.md Normal file
View File

@@ -0,0 +1,256 @@
# ES-Demo
OpenSearch/Elasticsearch 客户端实验项目,用于测试和验证 AWS OpenSearch Service 的各种功能和接口。
## 项目概述
这是一个用于实验 AWS OpenSearch Service 的 Go 项目。项目的主要目标是:
- 测试 OpenSearch 客户端的各种接口和功能
- 实现生产级别的代码质量
- 保持代码结构清晰、模块化
- 完备的测试覆盖
- 为未来转化为可复用的依赖库做准备
## 项目特点
- **生产级代码质量**:所有代码遵循 Go 官方最佳实践
- **模块化设计**:清晰的目录结构和职责划分
- **AI 辅助开发**:大部分代码由 AI 辅助生成,配合详细的指令文件
- **测试完备**:包含单元测试和集成测试
- **最新依赖**:使用最新版本的依赖库(除非有特殊需求)
## 技术栈
- **语言**: Go 1.25.3
- **OpenSearch 客户端**: opensearch-go/v2 v2.3.0
- **AWS SDK**: aws-sdk-go-v2 (config, aws/signer/v4)
- **环境变量**: godotenv v1.5.1
- **代码质量**: golangci-lint v1.64.8
- **测试框架**: Go 标准库 testing
## 项目结构
```text
es-demo/
├── README.md # 项目说明文档
├── .env.example # 环境变量配置示例
├── .gitignore # Git 忽略文件配置
├── .golangci.yml # golangci-lint 配置
├── test.ps1 # 测试流水线脚本Windows
├── go.mod # Go 模块依赖
├── go.sum # 依赖校验和
├── main.go # 程序入口
├── .github/
│ └── instructions/ # AI Agent 开发指令
│ ├── CLAUDE_GUIDELINE.instructions.md
│ └── DEV_GUIDELINE.instructions.md
├── config/ # 配置相关
│ ├── config.go # 全局配置,支持 .env 文件加载
│ └── config_test.go # 配置测试100% 覆盖率)
├── client/ # OpenSearch 客户端封装
│ ├── client.go # 客户端连接管理和 AWS 认证
│ └── client_test.go # 客户端测试
└── operations/ # 业务操作实现
├── cluster/ # 集群级别运维操作
│ ├── cluster.go # 集群信息查询
│ └── cluster_test.go # 集群操作测试
└── index/ # 索引级别业务操作
├── template.go # 索引模板管理CRUD
└── template_test.go # 模板操作测试
```
## 开发原则
### 代码质量要求
1. **遵循 Go 官方最佳实践**
- 使用 `gofmt` 格式化代码
- 遵循 [Effective Go](https://go.dev/doc/effective_go) 指南
- 遵循 [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments)
- 使用 `go vet` 进行静态检查
- 使用 `golangci-lint` 进行代码检查
2. **命名规范**
- 包名:简短、小写、单数形式
- 接口名:单个方法接口以 `-er` 结尾
- 变量名:驼峰命名,缩写词全大写(如 `URL``ID`
- 常量:驼峰命名或全大写下划线分隔
3. **错误处理**
- 所有错误必须处理
- 使用有意义的错误信息
- 适当时使用 `errors.Is()``errors.As()`
- 可考虑使用 `fmt.Errorf()` 包装错误
4. **注释规范**
- 所有导出的类型、函数必须有注释
- 注释以类型或函数名开头
- 包注释放在 `package` 语句之前
### 非功能需求(起步阶段)
由于项目处于实验阶段,以下方面采用简化实现:
- **配置管理**:使用全局变量,简单的配置结构
- **日志系统**:基础的日志输出,可使用全局 logger
- **错误处理**:基本的错误返回和处理,暂不需要复杂的错误链
- **性能优化**:功能优先,性能问题后续优化
这些简化不影响代码的可读性和可维护性,为后续改进预留空间。
## 功能模块
### 已完成功能
#### 客户端连接 (client/)
- [x] OpenSearch 客户端初始化和连接管理
- [x] AWS v4 签名认证
- [x] 配置验证和错误处理
- [x] 支持 .env 文件配置加载
#### 集群运维操作 (operations/cluster/)
- [x] 获取集群信息GetInfo
- 集群名称、版本、UUID
- 节点数、分片数统计
#### 索引业务操作 (operations/index/)
- [x] 索引模板管理Index Templates
- PutTemplate: 创建/更新索引模板
- GetTemplate: 获取指定模板配置
- DeleteTemplate: 删除索引模板
- ListTemplates: 列出所有模板
- 模板配置验证
### 待实现功能
以下功能将根据实际需求逐步实现:
- [ ] 索引管理(创建、删除、更新索引)
- [ ] 文档操作CRUD
- [ ] 搜索功能(基础查询、复杂查询)
- [ ] 聚合查询
- [ ] 批量操作
- [ ] 性能测试工具
## 快速开始
### 环境要求
- Go 1.25.3 或更高版本
- AWS 账号和 OpenSearch Service 实例
- AWS 访问凭证Access Key ID 和 Secret Access Key
### 安装依赖
```bash
go mod download
```
### 配置
[配置说明待补充]
### 运行
```bash
go run main.go
```
### 测试
#### 测试流水线(推荐)
使用自动化测试流水线,包含 Linting、Build、Test 和 Cleanup 四个阶段:
```powershell
# Windows PowerShell
powershell -ExecutionPolicy Bypass -File .\test.ps1
# 跳过特定阶段
powershell -ExecutionPolicy Bypass -File .\test.ps1 -SkipLint
powershell -ExecutionPolicy Bypass -File .\test.ps1 -SkipBuild
```
#### 手动测试
```bash
# 代码质量检查
golangci-lint run ./...
# 编译检查
go build -o NUL .
# 运行所有测试(跳过集成测试)
go test -v -short ./...
# 运行测试并显示覆盖率
go test -v -short -coverprofile=coverage.out ./...
go tool cover -func=coverage.out
# 生成 HTML 覆盖率报告
go tool cover -html=coverage.out -o coverage.html
```
**注意**: 使用 `-short` 标志跳过需要真实 OpenSearch 实例的集成测试。
## AI 辅助开发
本项目大量使用 AI 辅助开发,遵循严格的 TDD测试驱动开发流程。详细的开发指令请参考
- [开发准则](./.github/instructions/DEV_GUIDELINE.instructions.md) - Go 最佳实践、TDD 流程、测试流水线要求
- [Agent 执行协议](./.github/instructions/CLAUDE_GUIDELINE.instructions.md) - Agent 工作流程和执行约束
## 开发路线图
### 第一阶段:基础设施(已完成)
- [x] 项目初始化
- [x] 基础项目结构Client + Operations 架构)
- [x] OpenSearch 客户端封装连接管理、AWS 认证)
- [x] 基本测试框架(单元测试、集成测试分离)
- [x] 测试流水线自动化Lint → Build → Test → Cleanup
- [x] 代码质量保障golangci-lint 集成)
### 第二阶段:核心功能(进行中)
- [x] 集群运维操作GetInfo
- [x] 索引模板管理CRUD
- [ ] 索引管理功能(创建、删除、更新索引)
- [ ] 文档 CRUD 操作
- [ ] 基础搜索功能
- [ ] 完善测试覆盖
### 第三阶段:高级功能
- [ ] 聚合查询
- [ ] 批量操作优化
- [ ] 性能测试工具
- [ ] 错误处理优化
- [ ] 重试和容错机制
### 第四阶段:库化准备
- [ ] API 设计优化
- [ ] 完整文档
- [ ] 示例代码
- [ ] 发布为可复用库
## 贡献指南
由于本项目主要用于个人实验,暂不接受外部贡献。但欢迎提出建议和问题。
## 许可证
[待定]
## 联系方式
[待补充]
---
**注意**: 本项目处于积极开发中API 可能会发生变化。

135
client/client.go Normal file
View File

@@ -0,0 +1,135 @@
// Package client provides a high-level interface for interacting with AWS OpenSearch Service.
// It handles connection management, authentication, and request signing.
package client
import (
"context"
"crypto/tls"
"errors"
"fmt"
"net/http"
"time"
"github.com/aws/aws-sdk-go-v2/aws"
v4 "github.com/aws/aws-sdk-go-v2/aws/signer/v4"
"github.com/opensearch-project/opensearch-go/v2"
requestsigner "github.com/opensearch-project/opensearch-go/v2/signer/awsv2"
)
var (
// ErrInvalidConfig is returned when the client configuration is invalid.
ErrInvalidConfig = errors.New("invalid configuration")
// ErrRequestFailed is returned when a request to OpenSearch fails.
ErrRequestFailed = errors.New("request failed")
)
const (
// DefaultTimeout is the default timeout for requests.
DefaultTimeout = 30 * time.Second
)
// Config holds the configuration for the OpenSearch client.
type Config struct {
// Endpoint is the OpenSearch cluster endpoint URL.
Endpoint string
// Region is the AWS region where the OpenSearch cluster is located.
Region string
// AccessKey is the AWS access key ID for authentication.
AccessKey string
// SecretKey is the AWS secret access key for authentication.
SecretKey string
// Timeout is the request timeout duration. Defaults to DefaultTimeout if not set.
Timeout time.Duration
}
// Client represents an OpenSearch client that manages connections.
type Client struct {
config *Config
client *opensearch.Client
signer *v4.Signer
}
// NewClient creates a new OpenSearch client with the given configuration.
// It returns an error if the configuration is invalid or if the client cannot be initialized.
func NewClient(cfg *Config) (*Client, error) {
if cfg == nil {
return nil, fmt.Errorf("%w: config cannot be nil", ErrInvalidConfig)
}
if err := validateConfig(cfg); err != nil {
return nil, fmt.Errorf("%w: %v", ErrInvalidConfig, err)
}
// TODO: Use cfg.Timeout for client configuration
// Currently timeout is not directly supported by opensearch-go client config
// Create AWS credentials
awsConfig := aws.Config{
Region: cfg.Region,
Credentials: aws.CredentialsProviderFunc(func(ctx context.Context) (aws.Credentials, error) {
return aws.Credentials{
AccessKeyID: cfg.AccessKey,
SecretAccessKey: cfg.SecretKey,
}, nil
}),
}
// Create AWS v4 signer
signer := v4.NewSigner()
// Create request signer for OpenSearch
awsSigner, err := requestsigner.NewSignerWithService(awsConfig, "es")
if err != nil {
return nil, fmt.Errorf("failed to create request signer: %w", err)
}
// Configure OpenSearch client
osConfig := opensearch.Config{
Addresses: []string{cfg.Endpoint},
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
MinVersion: tls.VersionTLS12,
},
},
Signer: awsSigner,
}
// Create OpenSearch client
osClient, err := opensearch.NewClient(osConfig)
if err != nil {
return nil, fmt.Errorf("failed to create opensearch client: %w", err)
}
return &Client{
config: cfg,
client: osClient,
signer: signer,
}, nil
}
// GetClient returns the underlying OpenSearch client for use by operation packages.
func (c *Client) GetClient() *opensearch.Client {
return c.client
}
// validateConfig validates the client configuration.
func validateConfig(cfg *Config) error {
if cfg.Endpoint == "" {
return errors.New("endpoint is required")
}
if cfg.Region == "" {
return errors.New("region is required")
}
if cfg.AccessKey == "" {
return errors.New("access key is required")
}
if cfg.SecretKey == "" {
return errors.New("secret key is required")
}
return nil
}

113
client/client_test.go Normal file
View File

@@ -0,0 +1,113 @@
package client
import (
"errors"
"testing"
"time"
)
func TestNewClient(t *testing.T) {
tests := []struct {
name string
config *Config
wantErr error
}{
{
name: "valid config",
config: &Config{
Endpoint: "https://example.com",
Region: "us-east-1",
AccessKey: "test-key",
SecretKey: "test-secret",
},
wantErr: nil,
},
{
name: "nil config",
config: nil,
wantErr: ErrInvalidConfig,
},
{
name: "missing endpoint",
config: &Config{
Region: "us-east-1",
AccessKey: "test-key",
SecretKey: "test-secret",
},
wantErr: ErrInvalidConfig,
},
{
name: "missing region",
config: &Config{
Endpoint: "https://example.com",
AccessKey: "test-key",
SecretKey: "test-secret",
},
wantErr: ErrInvalidConfig,
},
{
name: "missing access key",
config: &Config{
Endpoint: "https://example.com",
Region: "us-east-1",
SecretKey: "test-secret",
},
wantErr: ErrInvalidConfig,
},
{
name: "missing secret key",
config: &Config{
Endpoint: "https://example.com",
Region: "us-east-1",
AccessKey: "test-key",
},
wantErr: ErrInvalidConfig,
},
{
name: "with custom timeout",
config: &Config{
Endpoint: "https://example.com",
Region: "us-east-1",
AccessKey: "test-key",
SecretKey: "test-secret",
Timeout: 60 * time.Second,
},
wantErr: nil,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := NewClient(tt.config)
if !errors.Is(err, tt.wantErr) {
t.Errorf("NewClient() error = %v, wantErr %v", err, tt.wantErr)
return
}
if tt.wantErr == nil && got == nil {
t.Error("NewClient() returned nil client with no error")
}
if tt.wantErr != nil && got != nil {
t.Error("NewClient() returned client with error")
}
})
}
}
func TestClient_GetClient(t *testing.T) {
cfg := &Config{
Endpoint: "https://example.com",
Region: "us-east-1",
AccessKey: "test-key",
SecretKey: "test-secret",
}
client, err := NewClient(cfg)
if err != nil {
t.Fatalf("failed to create client: %v", err)
}
osClient := client.GetClient()
if osClient == nil {
t.Error("GetClient() returned nil")
}
}

47
config/config.go Normal file
View File

@@ -0,0 +1,47 @@
// Package config provides global configuration management for the application.
// This is a simplified implementation suitable for the experimental phase.
package config
import (
"os"
"github.com/joho/godotenv"
)
var (
// Endpoint is the OpenSearch cluster endpoint URL.
Endpoint string
// Region is the AWS region where the OpenSearch cluster is located.
Region string
// AccessKey is the AWS access key ID for authentication.
AccessKey string
// SecretKey is the AWS secret access key for authentication.
SecretKey string
)
// Load loads configuration from a .env file and then from environment variables.
// If the file does not exist, it only loads from environment variables without error.
func Load(filePath string) error {
// Load .env file if it exists
// This will set environment variables from the file
_ = godotenv.Load(filePath)
// Initialize global config from environment variables
Endpoint = os.Getenv("ES_ENDPOINT")
AccessKey = os.Getenv("AWS_ACCESS_KEY_ID")
SecretKey = os.Getenv("AWS_SECRET_ACCESS_KEY")
Region = os.Getenv("AWS_REGION")
return nil
}
// Init initializes the global configuration from environment variables.
func Init() {
Endpoint = os.Getenv("ES_ENDPOINT")
AccessKey = os.Getenv("AWS_ACCESS_KEY_ID")
SecretKey = os.Getenv("AWS_SECRET_ACCESS_KEY")
Region = os.Getenv("AWS_REGION")
}

114
config/config_test.go Normal file
View File

@@ -0,0 +1,114 @@
package config
import (
"os"
"testing"
)
func TestInit(t *testing.T) {
// Save original env vars
origEndpoint := os.Getenv("ES_ENDPOINT")
origRegion := os.Getenv("AWS_REGION")
origAccessKey := os.Getenv("AWS_ACCESS_KEY_ID")
origSecretKey := os.Getenv("AWS_SECRET_ACCESS_KEY")
// Restore original env vars after test
defer func() {
os.Setenv("ES_ENDPOINT", origEndpoint)
os.Setenv("AWS_REGION", origRegion)
os.Setenv("AWS_ACCESS_KEY_ID", origAccessKey)
os.Setenv("AWS_SECRET_ACCESS_KEY", origSecretKey)
}()
// Set test env vars
os.Setenv("ES_ENDPOINT", "https://test.example.com")
os.Setenv("AWS_REGION", "us-west-2")
os.Setenv("AWS_ACCESS_KEY_ID", "test-access-key")
os.Setenv("AWS_SECRET_ACCESS_KEY", "test-secret-key")
// Test Init
Init()
if Endpoint != "https://test.example.com" {
t.Errorf("Endpoint = %v, want %v", Endpoint, "https://test.example.com")
}
if Region != "us-west-2" {
t.Errorf("Region = %v, want %v", Region, "us-west-2")
}
if AccessKey != "test-access-key" {
t.Errorf("AccessKey = %v, want %v", AccessKey, "test-access-key")
}
if SecretKey != "test-secret-key" {
t.Errorf("SecretKey = %v, want %v", SecretKey, "test-secret-key")
}
}
func TestLoad(t *testing.T) {
// Save original env vars
origEndpoint := os.Getenv("ES_ENDPOINT")
origRegion := os.Getenv("AWS_REGION")
origAccessKey := os.Getenv("AWS_ACCESS_KEY_ID")
origSecretKey := os.Getenv("AWS_SECRET_ACCESS_KEY")
// Clear environment variables before test
os.Unsetenv("ES_ENDPOINT")
os.Unsetenv("AWS_REGION")
os.Unsetenv("AWS_ACCESS_KEY_ID")
os.Unsetenv("AWS_SECRET_ACCESS_KEY")
// Restore original env vars after test
defer func() {
if origEndpoint != "" {
os.Setenv("ES_ENDPOINT", origEndpoint)
}
if origRegion != "" {
os.Setenv("AWS_REGION", origRegion)
}
if origAccessKey != "" {
os.Setenv("AWS_ACCESS_KEY_ID", origAccessKey)
}
if origSecretKey != "" {
os.Setenv("AWS_SECRET_ACCESS_KEY", origSecretKey)
}
// Reload original config
Init()
}()
// Create a temporary .env file
tmpFile := ".env.test"
content := `ES_ENDPOINT=https://test-load.example.com
AWS_REGION=eu-west-1
AWS_ACCESS_KEY_ID=load-access-key
AWS_SECRET_ACCESS_KEY=load-secret-key
`
if err := os.WriteFile(tmpFile, []byte(content), 0644); err != nil {
t.Fatalf("failed to create test .env file: %v", err)
}
defer os.Remove(tmpFile)
// Test Load
if err := Load(tmpFile); err != nil {
t.Fatalf("Load() error = %v", err)
}
if Endpoint != "https://test-load.example.com" {
t.Errorf("Endpoint = %v, want %v", Endpoint, "https://test-load.example.com")
}
if Region != "eu-west-1" {
t.Errorf("Region = %v, want %v", Region, "eu-west-1")
}
if AccessKey != "load-access-key" {
t.Errorf("AccessKey = %v, want %v", AccessKey, "load-access-key")
}
if SecretKey != "load-secret-key" {
t.Errorf("SecretKey = %v, want %v", SecretKey, "load-secret-key")
}
}
func TestLoad_FileNotFound(t *testing.T) {
// Test with non-existent file - should not return error
err := Load("non-existent-file.env")
if err != nil {
t.Errorf("Load() with non-existent file should not error, got %v", err)
}
}

14
go.mod Normal file
View File

@@ -0,0 +1,14 @@
module es-demo
go 1.25.3
require (
github.com/aws/aws-sdk-go-v2 v1.39.6
github.com/joho/godotenv v1.5.1
github.com/opensearch-project/opensearch-go/v2 v2.3.0
)
require (
github.com/aws/aws-sdk-go-v2/config v1.31.20 // indirect
github.com/aws/smithy-go v1.23.2 // indirect
)

95
go.sum Normal file
View File

@@ -0,0 +1,95 @@
github.com/aws/aws-sdk-go v1.44.263/go.mod h1:aVsgQcEevwlmQ7qHE9I3h+dtQgpqhFB+i8Phjh7fkwI=
github.com/aws/aws-sdk-go-v2 v1.18.0/go.mod h1:uzbQtefpm44goOPmdKyAlXSNcwlRgF3ePWVW6EtJvvw=
github.com/aws/aws-sdk-go-v2 v1.39.6 h1:2JrPCVgWJm7bm83BDwY5z8ietmeJUbh3O2ACnn+Xsqk=
github.com/aws/aws-sdk-go-v2 v1.39.6/go.mod h1:c9pm7VwuW0UPxAEYGyTmyurVcNrbF6Rt/wixFqDhcjE=
github.com/aws/aws-sdk-go-v2/config v1.18.25/go.mod h1:dZnYpD5wTW/dQF0rRNLVypB396zWCcPiBIvdvSWHEg4=
github.com/aws/aws-sdk-go-v2/config v1.31.20 h1:/jWF4Wu90EhKCgjTdy1DGxcbcbNrjfBHvksEL79tfQc=
github.com/aws/aws-sdk-go-v2/config v1.31.20/go.mod h1:95Hh1Tc5VYKL9NJ7tAkDcqeKt+MCXQB1hQZaRdJIZE0=
github.com/aws/aws-sdk-go-v2/credentials v1.13.24/go.mod h1:jYPYi99wUOPIFi0rhiOvXeSEReVOzBqFNOX5bXYoG2o=
github.com/aws/aws-sdk-go-v2/credentials v1.18.24 h1:iJ2FmPT35EaIB0+kMa6TnQ+PwG5A1prEdAw+PsMzfHg=
github.com/aws/aws-sdk-go-v2/credentials v1.18.24/go.mod h1:U91+DrfjAiXPDEGYhh/x29o4p0qHX5HDqG7y5VViv64=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.3/go.mod h1:4Q0UFP0YJf0NrsEuEYHpM9fTSEVnD16Z3uyEF7J9JGM=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 h1:T1brd5dR3/fzNFAQch/iBKeX07/ffu/cLu+q+RuzEWk=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13/go.mod h1:Peg/GBAQ6JDt+RoBf4meB1wylmAipb7Kg2ZFakZTlwk=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.33/go.mod h1:7i0PF1ME/2eUPFcjkVIwq+DOygHEoK92t5cDqNgYbIw=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 h1:a+8/MLcWlIxo1lF9xaGt3J/u3yOZx+CdSveSNwjhD40=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13/go.mod h1:oGnKwIYZ4XttyU2JWxFrwvhF6YKiK/9/wmE3v3Iu9K8=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.27/go.mod h1:UrHnn3QV/d0pBZ6QBAEQcqFLf8FAzLmoUfPVIueOvoM=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 h1:HBSI2kDkMdWz4ZM7FjwE7e/pWDEZ+nR95x8Ztet1ooY=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13/go.mod h1:YE94ZoDArI7awZqJzBAZ3PDD2zSfuP7w6P2knOzIn8M=
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.34/go.mod h1:Etz2dj6UHYuw+Xw830KfzCfWGMzqvUTCjUj5b76GVDc=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 h1:WKuaxf++XKWlHWu9ECbMlha8WOEGm0OUEZqm4K/Gcfk=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4/go.mod h1:ZWy7j6v1vWGmPReu0iSGvRiise4YI5SkR3OHKTZ6Wuc=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 h1:x2Ibm/Af8Fi+BH+Hsn9TXGdT+hKbDd5XOTZxTMxDk7o=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3/go.mod h1:IW1jwyrQgMdhisceG8fQLmQIydcT/jWY21rFhzgaKwo=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.27/go.mod h1:EOwBD4J4S5qYszS5/3DpkejfuK+Z5/1uzICfPaZLtqw=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 h1:kDqdFvMY4AtKoACfzIGD8A0+hbT41KTKF//gq7jITfM=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13/go.mod h1:lmKuogqSU3HzQCwZ9ZtcqOc5XGMqtDK7OIc2+DxiUEg=
github.com/aws/aws-sdk-go-v2/service/sso v1.12.10/go.mod h1:ouy2P4z6sJN70fR3ka3wD3Ro3KezSxU6eKGQI2+2fjI=
github.com/aws/aws-sdk-go-v2/service/sso v1.30.3 h1:NjShtS1t8r5LUfFVtFeI8xLAHQNTa7UI0VawXlrBMFQ=
github.com/aws/aws-sdk-go-v2/service/sso v1.30.3/go.mod h1:fKvyjJcz63iL/ftA6RaM8sRCtN4r4zl4tjL3qw5ec7k=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.10/go.mod h1:AFvkxc8xfBe8XA+5St5XIHHrQQtkxqrRincx4hmMHOk=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.7 h1:gTsnx0xXNQ6SBbymoDvcoRHL+q4l/dAFsQuKfDWSaGc=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.7/go.mod h1:klO+ejMvYsB4QATfEOIXk8WAEwN4N0aBfJpvC+5SZBo=
github.com/aws/aws-sdk-go-v2/service/sts v1.19.0/go.mod h1:BgQOMsg8av8jset59jelyPW7NoZcZXLVpDsXunGDrk8=
github.com/aws/aws-sdk-go-v2/service/sts v1.40.2 h1:HK5ON3KmQV2HcAunnx4sKLB9aPf3gKGwVAf7xnx0QT0=
github.com/aws/aws-sdk-go-v2/service/sts v1.40.2/go.mod h1:E19xDjpzPZC7LS2knI9E6BaRFDK43Eul7vd6rSq2HWk=
github.com/aws/smithy-go v1.13.5/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
github.com/aws/smithy-go v1.23.2 h1:Crv0eatJUQhaManss33hS5r40CG3ZFH+21XSkqMrIUM=
github.com/aws/smithy-go v1.23.2/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/opensearch-project/opensearch-go/v2 v2.3.0 h1:nQIEMr+A92CkhHrZgUhcfsrZjibvB3APXf2a1VwCmMQ=
github.com/opensearch-project/opensearch-go/v2 v2.3.0/go.mod h1:8LDr9FCgUTVoT+5ESjc2+iaZuldqE+23Iq0r1XeNue8=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.1.0/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

109
main.go Normal file
View File

@@ -0,0 +1,109 @@
package main
import (
"context"
"fmt"
"log"
"time"
"es-demo/client"
"es-demo/config"
"es-demo/operations/cluster"
"es-demo/operations/index"
)
func main() {
// Load configuration from .env file and environment variables
if err := config.Load(".env"); err != nil {
log.Fatalf("Failed to load configuration: %v", err)
}
// Create client configuration
cfg := &client.Config{
Endpoint: config.Endpoint,
Region: config.Region,
AccessKey: config.AccessKey,
SecretKey: config.SecretKey,
Timeout: 30 * time.Second,
}
// Create OpenSearch client
osClient, err := client.NewClient(cfg)
if err != nil {
log.Fatalf("Failed to create OpenSearch client: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// Get cluster information
info, err := cluster.GetInfo(ctx, osClient)
if err != nil {
log.Fatalf("Failed to get cluster info: %v", err)
}
// Display cluster information
fmt.Printf("=== Cluster Information ===\n")
fmt.Printf("Cluster Name: %s\n", info.ClusterName)
fmt.Printf("Cluster UUID: %s\n", info.ClusterUUID)
fmt.Printf("Version: %s\n", info.Version.Number)
fmt.Printf("Lucene Version: %s\n", info.Version.LuceneVersion)
fmt.Printf("Tagline: %s\n\n", info.Tagline)
// Example: Create an index template
template := &index.Template{
IndexPatterns: []string{"logs-*"},
Settings: map[string]any{
"number_of_shards": 1,
"number_of_replicas": 1,
},
Mappings: map[string]any{
"properties": map[string]any{
"timestamp": map[string]any{
"type": "date",
},
"message": map[string]any{
"type": "text",
},
},
},
Priority: 100,
}
fmt.Printf("=== Index Template Operations ===\n")
// Put template
if err := index.PutTemplate(ctx, osClient, "logs-template", template); err != nil {
log.Printf("Warning: Failed to create template: %v\n", err)
} else {
fmt.Printf("✓ Created template: logs-template\n")
}
// Get template
retrievedTemplate, err := index.GetTemplate(ctx, osClient, "logs-template")
if err != nil {
log.Printf("Warning: Failed to get template: %v\n", err)
} else {
fmt.Printf("✓ Retrieved template: %+v\n", retrievedTemplate.IndexPatterns)
}
// List templates
templates, err := index.ListTemplates(ctx, osClient)
if err != nil {
log.Printf("Warning: Failed to list templates: %v\n", err)
} else {
fmt.Printf("✓ Total templates: %d\n", len(templates))
for name := range templates {
fmt.Printf(" - %s\n", name)
}
}
// Delete template (commented out to avoid cleanup in demo)
// if err := index.DeleteTemplate(ctx, osClient, "logs-template"); err != nil {
// log.Printf("Warning: Failed to delete template: %v\n", err)
// } else {
// fmt.Printf("✓ Deleted template: logs-template\n")
// }
fmt.Printf("\nDemo completed successfully!\n")
}

View File

@@ -0,0 +1,58 @@
// Package cluster provides cluster-level operations for OpenSearch administration.
package cluster
import (
"context"
"encoding/json"
"fmt"
"os"
"es-demo/client"
"github.com/opensearch-project/opensearch-go/v2/opensearchapi"
)
// Info represents essential information about the OpenSearch cluster.
type Info struct {
Name string `json:"name"`
ClusterName string `json:"cluster_name"`
ClusterUUID string `json:"cluster_uuid"`
Version struct {
Number string `json:"number"`
BuildType string `json:"build_type"`
BuildHash string `json:"build_hash"`
BuildDate string `json:"build_date"`
BuildSnapshot bool `json:"build_snapshot"`
LuceneVersion string `json:"lucene_version"`
MinimumWireCompatibilityVersion string `json:"minimum_wire_compatibility_version"`
MinimumIndexCompatibilityVersion string `json:"minimum_index_compatibility_version"`
} `json:"version"`
Tagline string `json:"tagline"`
}
// GetInfo retrieves essential information about the OpenSearch cluster.
// It returns the cluster information or an error if the request fails.
func GetInfo(ctx context.Context, c *client.Client) (*Info, error) {
req := opensearchapi.InfoRequest{}
res, err := req.Do(ctx, c.GetClient())
if err != nil {
return nil, fmt.Errorf("failed to execute info request: %w", err)
}
defer func() {
if closeErr := res.Body.Close(); closeErr != nil {
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
}
}()
if res.IsError() {
return nil, fmt.Errorf("request failed with status: %s", res.Status())
}
var info Info
if err := json.NewDecoder(res.Body).Decode(&info); err != nil {
return nil, fmt.Errorf("failed to decode cluster info: %w", err)
}
return &info, nil
}

View File

@@ -0,0 +1,40 @@
package cluster
import (
"context"
"testing"
"time"
"es-demo/client"
)
func TestGetInfo(t *testing.T) {
// This is a unit test that doesn't require real connection
// Integration tests should be run separately
if testing.Short() {
t.Skip("skipping integration test in short mode")
}
cfg := &client.Config{
Endpoint: "https://example.com",
Region: "us-east-1",
AccessKey: "test-key",
SecretKey: "test-secret",
}
c, err := client.NewClient(cfg)
if err != nil {
t.Fatalf("failed to create client: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// This will fail without a real cluster, which is expected in unit tests
_, err = GetInfo(ctx, c)
// We just verify the method signature is correct
// Real integration tests would check actual responses
if err == nil {
t.Log("GetInfo succeeded (unexpected in unit test)")
}
}

View File

@@ -0,0 +1,217 @@
// Package index provides index-level operations for OpenSearch.
package index
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"es-demo/client"
"github.com/opensearch-project/opensearch-go/v2/opensearchapi"
)
var (
// ErrInvalidTemplate is returned when the template configuration is invalid.
ErrInvalidTemplate = errors.New("invalid template configuration")
// ErrTemplateNotFound is returned when the specified template does not exist.
ErrTemplateNotFound = errors.New("template not found")
)
// Template represents an OpenSearch index template.
type Template struct {
// IndexPatterns defines the index patterns this template applies to.
IndexPatterns []string `json:"index_patterns"`
// Settings contains index settings like number of shards and replicas.
Settings map[string]any `json:"settings,omitempty"`
// Mappings defines the field mappings for the index.
Mappings map[string]any `json:"mappings,omitempty"`
// Aliases defines index aliases.
Aliases map[string]any `json:"aliases,omitempty"`
// Priority determines template precedence when multiple templates match.
Priority int `json:"priority,omitempty"`
// Version is used for external version management.
Version int `json:"version,omitempty"`
}
// TemplateResponse represents the response when getting a template.
type TemplateResponse struct {
IndexTemplates []struct {
Name string `json:"name"`
IndexTemplate Template `json:"index_template"`
} `json:"index_templates"`
}
// PutTemplate creates or updates an index template.
func PutTemplate(ctx context.Context, c *client.Client, name string, template *Template) error {
if err := validateTemplate(template); err != nil {
return err
}
if name == "" {
return fmt.Errorf("%w: template name is required", ErrInvalidTemplate)
}
body, err := json.Marshal(template)
if err != nil {
return fmt.Errorf("failed to marshal template: %w", err)
}
req := opensearchapi.IndicesPutIndexTemplateRequest{
Name: name,
Body: bytes.NewReader(body),
}
res, err := req.Do(ctx, c.GetClient())
if err != nil {
return fmt.Errorf("failed to execute put template request: %w", err)
}
defer func() {
if closeErr := res.Body.Close(); closeErr != nil {
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
}
}()
if res.IsError() {
bodyBytes, _ := io.ReadAll(res.Body)
return fmt.Errorf("put template failed with status %s: %s", res.Status(), string(bodyBytes))
}
return nil
}
// GetTemplate retrieves an index template by name.
func GetTemplate(ctx context.Context, c *client.Client, name string) (*Template, error) {
if name == "" {
return nil, fmt.Errorf("%w: template name is required", ErrInvalidTemplate)
}
req := opensearchapi.IndicesGetIndexTemplateRequest{
Name: []string{name},
}
res, err := req.Do(ctx, c.GetClient())
if err != nil {
return nil, fmt.Errorf("failed to execute get template request: %w", err)
}
defer func() {
if closeErr := res.Body.Close(); closeErr != nil {
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
}
}()
if res.IsError() {
if res.StatusCode == 404 {
return nil, ErrTemplateNotFound
}
bodyBytes, _ := io.ReadAll(res.Body)
return nil, fmt.Errorf("get template failed with status %s: %s", res.Status(), string(bodyBytes))
}
var response TemplateResponse
if err := json.NewDecoder(res.Body).Decode(&response); err != nil {
return nil, fmt.Errorf("failed to decode template response: %w", err)
}
if len(response.IndexTemplates) == 0 {
return nil, ErrTemplateNotFound
}
return &response.IndexTemplates[0].IndexTemplate, nil
}
// DeleteTemplate deletes an index template.
func DeleteTemplate(ctx context.Context, c *client.Client, name string) error {
if name == "" {
return fmt.Errorf("%w: template name is required", ErrInvalidTemplate)
}
req := opensearchapi.IndicesDeleteIndexTemplateRequest{
Name: name,
}
res, err := req.Do(ctx, c.GetClient())
if err != nil {
return fmt.Errorf("failed to execute delete template request: %w", err)
}
defer func() {
if closeErr := res.Body.Close(); closeErr != nil {
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
}
}()
if res.IsError() {
if res.StatusCode == 404 {
return ErrTemplateNotFound
}
bodyBytes, _ := io.ReadAll(res.Body)
return fmt.Errorf("delete template failed with status %s: %s", res.Status(), string(bodyBytes))
}
return nil
}
// ListTemplates retrieves all index templates or templates matching a pattern.
func ListTemplates(ctx context.Context, c *client.Client, names ...string) (map[string]*Template, error) {
var nameList []string
if len(names) > 0 {
nameList = names
}
req := opensearchapi.IndicesGetIndexTemplateRequest{
Name: nameList,
}
res, err := req.Do(ctx, c.GetClient())
if err != nil {
return nil, fmt.Errorf("failed to execute list templates request: %w", err)
}
defer func() {
if closeErr := res.Body.Close(); closeErr != nil {
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
}
}()
if res.IsError() {
if res.StatusCode == 404 {
return make(map[string]*Template), nil
}
bodyBytes, _ := io.ReadAll(res.Body)
return nil, fmt.Errorf("list templates failed with status %s: %s", res.Status(), string(bodyBytes))
}
var response TemplateResponse
if err := json.NewDecoder(res.Body).Decode(&response); err != nil {
return nil, fmt.Errorf("failed to decode templates response: %w", err)
}
templates := make(map[string]*Template)
for _, item := range response.IndexTemplates {
templates[item.Name] = &item.IndexTemplate
}
return templates, nil
}
// validateTemplate validates the template configuration.
func validateTemplate(template *Template) error {
if template == nil {
return fmt.Errorf("%w: template cannot be nil", ErrInvalidTemplate)
}
if len(template.IndexPatterns) == 0 {
return fmt.Errorf("%w: index patterns are required", ErrInvalidTemplate)
}
return nil
}

View File

@@ -0,0 +1,180 @@
package index
import (
"context"
"errors"
"testing"
"time"
"es-demo/client"
)
func TestPutTemplate(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test in short mode")
}
cfg := &client.Config{
Endpoint: "https://example.com",
Region: "us-east-1",
AccessKey: "test-key",
SecretKey: "test-secret",
}
c, err := client.NewClient(cfg)
if err != nil {
t.Fatalf("failed to create client: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
template := &Template{
IndexPatterns: []string{"test-*"},
Settings: map[string]any{
"number_of_shards": 1,
"number_of_replicas": 0,
},
Mappings: map[string]any{
"properties": map[string]any{
"timestamp": map[string]any{
"type": "date",
},
},
},
}
// This will fail without a real cluster
err = PutTemplate(ctx, c, "test-template", template)
if err == nil {
t.Log("PutTemplate succeeded (unexpected in unit test)")
}
}
func TestGetTemplate(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test in short mode")
}
cfg := &client.Config{
Endpoint: "https://example.com",
Region: "us-east-1",
AccessKey: "test-key",
SecretKey: "test-secret",
}
c, err := client.NewClient(cfg)
if err != nil {
t.Fatalf("failed to create client: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// This will fail without a real cluster
_, err = GetTemplate(ctx, c, "test-template")
if err == nil {
t.Log("GetTemplate succeeded (unexpected in unit test)")
}
}
func TestDeleteTemplate(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test in short mode")
}
cfg := &client.Config{
Endpoint: "https://example.com",
Region: "us-east-1",
AccessKey: "test-key",
SecretKey: "test-secret",
}
c, err := client.NewClient(cfg)
if err != nil {
t.Fatalf("failed to create client: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// This will fail without a real cluster
err = DeleteTemplate(ctx, c, "test-template")
if err == nil {
t.Log("DeleteTemplate succeeded (unexpected in unit test)")
}
}
func TestListTemplates(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test in short mode")
}
cfg := &client.Config{
Endpoint: "https://example.com",
Region: "us-east-1",
AccessKey: "test-key",
SecretKey: "test-secret",
}
c, err := client.NewClient(cfg)
if err != nil {
t.Fatalf("failed to create client: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// This will fail without a real cluster
_, err = ListTemplates(ctx, c)
if err == nil {
t.Log("ListTemplates succeeded (unexpected in unit test)")
}
}
func TestTemplate_Validate(t *testing.T) {
tests := []struct {
name string
template *Template
wantErr error
}{
{
name: "valid template",
template: &Template{
IndexPatterns: []string{"logs-*"},
Settings: map[string]any{
"number_of_shards": 1,
},
},
wantErr: nil,
},
{
name: "nil template",
template: nil,
wantErr: ErrInvalidTemplate,
},
{
name: "empty index patterns",
template: &Template{
IndexPatterns: []string{},
},
wantErr: ErrInvalidTemplate,
},
{
name: "nil index patterns",
template: &Template{
IndexPatterns: nil,
},
wantErr: ErrInvalidTemplate,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := validateTemplate(tt.template)
if !errors.Is(err, tt.wantErr) {
t.Errorf("validateTemplate() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}

213
test.ps1 Normal file
View File

@@ -0,0 +1,213 @@
# 测试流水线脚本
# 用途: 执行完整的代码质量检查流程
# 包括: Linting -> Build -> Test -> Coverage -> Cleanup
param(
[switch]$SkipLint,
[switch]$SkipBuild,
[switch]$SkipTest,
[switch]$Verbose
)
$ErrorActionPreference = "Stop"
$Script:FailureCount = 0
# 颜色输出函数
function Write-Success { param($Message) Write-Host "$Message" -ForegroundColor Green }
function Write-Failure { param($Message) Write-Host "$Message" -ForegroundColor Red; $Script:FailureCount++ }
function Write-Info { param($Message) Write-Host "$Message" -ForegroundColor Cyan }
function Write-Section { param($Message) Write-Host "`n=== $Message ===" -ForegroundColor Yellow }
# 清理函数
function Cleanup {
Write-Section "Cleanup"
# 删除构建产物
$artifacts = @("*.exe", "*.test", "coverage.out", "coverage.html")
foreach ($pattern in $artifacts) {
Get-ChildItem -Path . -Filter $pattern -Recurse -ErrorAction SilentlyContinue |
Remove-Item -Force -ErrorAction SilentlyContinue
}
# 清理测试生成的临时文件
Get-ChildItem -Path . -Filter ".env.test" -Recurse -ErrorAction SilentlyContinue |
Remove-Item -Force -ErrorAction SilentlyContinue
Write-Success "Cleanup completed"
}
# Linting
function Run-Linting {
Write-Section "Code Linting"
try {
Write-Info "Running golangci-lint..."
# 检查 golangci-lint 是否安装
$lintPath = Get-Command golangci-lint -ErrorAction SilentlyContinue
if (-not $lintPath) {
Write-Failure "golangci-lint not found. Please install it first."
return $false
}
# 运行 linter
$output = & golangci-lint run --config .golangci.yml ./... 2>&1
if ($LASTEXITCODE -eq 0) {
Write-Success "Linting passed"
return $true
} else {
Write-Failure "Linting failed"
if ($Verbose) {
Write-Host $output -ForegroundColor Red
}
return $false
}
}
catch {
Write-Failure "Linting error: $_"
return $false
}
}
# Build
function Run-Build {
Write-Section "Build"
try {
Write-Info "Building project..."
# 使用 NUL 设备作为输出Windows 的 /dev/null
$env:GOOS = ""
$env:GOARCH = ""
# 构建但不保存二进制文件
$output = go build -o NUL . 2>&1
if ($LASTEXITCODE -eq 0) {
Write-Success "Build successful"
return $true
} else {
Write-Failure "Build failed"
if ($Verbose) {
Write-Host $output -ForegroundColor Red
}
return $false
}
}
catch {
Write-Failure "Build error: $_"
return $false
}
}
# Tests
function Run-Tests {
Write-Section "Tests"
try {
Write-Info "Running unit tests..."
# 运行短测试(跳过集成测试)
# 注意: Windows 下不启用 race detector (需要 cgo)
# PowerShell 需要使用引号包裹带等号的参数
$testOutput = go test -v -short "-coverprofile=coverage.out" ./... 2>&1
if ($LASTEXITCODE -eq 0) {
Write-Success "All tests passed"
# 显示覆盖率摘要
Write-Info "Coverage summary:"
$coverageOutput = go tool cover -func coverage.out 2>&1
$totalLine = $coverageOutput | Select-String "total:"
if ($totalLine) {
Write-Host " $totalLine" -ForegroundColor White
}
# 可选:生成 HTML 覆盖率报告
if ($Verbose) {
go tool cover -html=coverage.out -o coverage.html
Write-Success "Coverage report generated: coverage.html"
}
return $true
} else {
Write-Failure "Tests failed"
if ($Verbose) {
Write-Host $output -ForegroundColor Red
}
return $false
}
}
catch {
Write-Failure "Test error: $_"
return $false
}
}
# 主流程
function Main {
Write-Host @"
ES-Demo Test Pipeline v1.0
"@ -ForegroundColor Magenta
$startTime = Get-Date
# 执行流水线步骤
$results = @{
Lint = $true
Build = $true
Test = $true
}
if (-not $SkipLint) {
$results.Lint = Run-Linting
} else {
Write-Info "Skipping linting"
}
if (-not $SkipBuild) {
$results.Build = Run-Build
} else {
Write-Info "Skipping build"
}
if (-not $SkipTest) {
$results.Test = Run-Tests
} else {
Write-Info "Skipping tests"
}
# 清理
Cleanup
# 结果汇总
$endTime = Get-Date
$duration = $endTime - $startTime
Write-Section "Summary"
Write-Host "Lint: " -NoNewline
if ($results.Lint) { Write-Success "PASS" } else { Write-Failure "FAIL" }
Write-Host "Build: " -NoNewline
if ($results.Build) { Write-Success "PASS" } else { Write-Failure "FAIL" }
Write-Host "Test: " -NoNewline
if ($results.Test) { Write-Success "PASS" } else { Write-Failure "FAIL" }
Write-Host "`nTotal time: $($duration.TotalSeconds) seconds" -ForegroundColor Cyan
# 返回退出码
if ($Script:FailureCount -gt 0) {
Write-Host "`n❌ Pipeline FAILED with $Script:FailureCount error(s)" -ForegroundColor Red
exit 1
} else {
Write-Host "`n✅ Pipeline PASSED" -ForegroundColor Green
exit 0
}
}
# 运行主流程
Main