feat: opensearch客户端初始化 feat: 索引模板接口 ai: 开发准则 chore: TDD流水线脚本
This commit is contained in:
14
.env.example
Normal file
14
.env.example
Normal file
@@ -0,0 +1,14 @@
|
||||
# OpenSearch 配置示例
|
||||
# 复制此文件为 .env 并填写实际值
|
||||
|
||||
# OpenSearch 集群端点
|
||||
ES_ENDPOINT=
|
||||
|
||||
# AWS 区域
|
||||
AWS_REGION=
|
||||
|
||||
# AWS 访问密钥 ID
|
||||
AWS_ACCESS_KEY_ID=
|
||||
|
||||
# AWS 访问密钥密文
|
||||
AWS_SECRET_ACCESS_KEY=
|
||||
23
.github/instructions/CLAUDE_GUIDELINE.instructions.md
vendored
Normal file
23
.github/instructions/CLAUDE_GUIDELINE.instructions.md
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
---
|
||||
applyTo: '**'
|
||||
---
|
||||
⚙️ AGENT 执行协议
|
||||
本协议定义了 Agent 在任务执行期间必须遵守的操作指南和工作流程约束。严格遵守所有要点是不可谈判的。
|
||||
|
||||
一、执行前强制检查
|
||||
强制审查开发准则:在启动任何开发或编码任务之前,你必须首先审查并确认遵守已经建立的开发准则(即系统 Prompt 或其他指令文件)。你所采取的任何行动都必须与这些总体规则保持一致。
|
||||
|
||||
二、设计与清晰度
|
||||
清晰度优先:如果用户需求含糊不清、不完整,或者关键逻辑不确定,你必须暂停执行并向用户提出澄清问题,以确保完全理解后再进行后续实现。
|
||||
|
||||
零成本抽象原则(KISS 原则):严格遵守“如无必要,勿增实体”的原则。不要引入任何对于所需功能并非严格必要的文件、模块、抽象或依赖。
|
||||
|
||||
三、开发工作流程(TDD)
|
||||
测试驱动开发(TDD):你必须严格遵循测试驱动开发(TDD)流程。任何新功能或修复的实现工作,只能在相应的、失败的测试用例已编写并保存之后才能开始。
|
||||
|
||||
四、输出与文件管理
|
||||
输出限制:禁止使用说明文档:你不得生成任何详细说明如何使用或调用所实现代码的补充文档。只应生成源代码和必要的配置文件。
|
||||
|
||||
文件生成限制:对于任何给定的模块或逻辑单元,你至多只能生成一个主要的文档文件(例如,该模块的 README.md 或设计文件)。避免在多个文件之间冗余地生成文档。
|
||||
|
||||
测试编译是否成功的输出文件:只是为了验证是否可以编译成功,不需要保留这些输出文件。
|
||||
975
.github/instructions/DEV_GUIDELINE.instructions.md
vendored
Normal file
975
.github/instructions/DEV_GUIDELINE.instructions.md
vendored
Normal file
@@ -0,0 +1,975 @@
|
||||
---
|
||||
applyTo: '**/*.go'
|
||||
---
|
||||
# AI Agent 开发指令
|
||||
|
||||
本文档为 AI Agent(GitHub Copilot)提供开发本项目时的详细指令和指南。
|
||||
|
||||
## 项目背景
|
||||
|
||||
这是一个用于测试和实验 AWS OpenSearch Service 的 Go 项目。项目目标是创建生产级别的代码,并为未来转化为可复用的依赖库做准备。
|
||||
|
||||
## 核心原则
|
||||
|
||||
### 1. 代码质量要求(最高优先级)
|
||||
|
||||
**必须遵循 Go 官方最佳实践:**
|
||||
|
||||
- **Effective Go**: https://go.dev/doc/effective_go
|
||||
- **Go Code Review Comments**: https://github.com/golang/go/wiki/CodeReviewComments
|
||||
- **Go 标准库风格**: 参考标准库的代码风格和设计模式
|
||||
|
||||
**具体要求:**
|
||||
|
||||
```go
|
||||
// ✅ 正确:导出的函数必须有文档注释,以函数名开头
|
||||
// NewClient creates and initializes a new OpenSearch client with the provided configuration.
|
||||
// It returns an error if the connection cannot be established.
|
||||
func NewClient(cfg *Config) (*Client, error) {
|
||||
// ...
|
||||
}
|
||||
|
||||
// ❌ 错误:缺少注释或注释不以函数名开头
|
||||
func NewClient(cfg *Config) (*Client, error) {
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
```go
|
||||
// ✅ 正确:错误处理不能忽略
|
||||
result, err := client.Search(ctx, query)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("search failed: %w", err)
|
||||
}
|
||||
|
||||
// ❌ 错误:忽略错误
|
||||
result, _ := client.Search(ctx, query)
|
||||
```
|
||||
|
||||
```go
|
||||
// ✅ 正确:使用有意义的变量名
|
||||
var maxRetryCount = 3
|
||||
var connectionTimeout = 30 * time.Second
|
||||
|
||||
// ❌ 错误:使用无意义的缩写
|
||||
var mrc = 3
|
||||
var ct = 30 * time.Second
|
||||
```
|
||||
|
||||
### 2. 命名规范
|
||||
|
||||
**包名:**
|
||||
- 小写、简短、单数形式
|
||||
- 避免使用下划线或驼峰命名
|
||||
- 示例:`client`、`search`、`document`(不是 `opensearch_client`)
|
||||
|
||||
**接口名:**
|
||||
- 单方法接口以 `-er` 结尾:`Reader`、`Writer`、`Searcher`
|
||||
- 多方法接口使用描述性名称:`IndexManager`、`DocumentService`
|
||||
|
||||
**变量和函数名:**
|
||||
- 驼峰命名(camelCase 或 PascalCase)
|
||||
- 缩写词保持一致大写:`URL`、`HTTP`、`ID`、`JSON`(不是 `Url`、`Http`、`Id`、`Json`)
|
||||
- 示例:`clientID`、`httpClient`、`parseURL`
|
||||
|
||||
**常量:**
|
||||
```go
|
||||
// ✅ 正确:使用驼峰命名
|
||||
const DefaultTimeout = 30 * time.Second
|
||||
const MaxRetries = 3
|
||||
|
||||
// 或者使用全大写(枚举类型)
|
||||
const (
|
||||
StatusPending = "PENDING"
|
||||
StatusComplete = "COMPLETE"
|
||||
)
|
||||
```
|
||||
|
||||
### 3. 错误处理规范
|
||||
|
||||
**基本错误处理:**
|
||||
|
||||
```go
|
||||
// ✅ 正确:包装错误,提供上下文信息
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create index %q: %w", indexName, err)
|
||||
}
|
||||
|
||||
// ✅ 正确:自定义错误类型(当需要时)
|
||||
var ErrIndexNotFound = errors.New("index not found")
|
||||
var ErrInvalidQuery = errors.New("invalid query syntax")
|
||||
|
||||
// ✅ 正确:错误检查使用 errors.Is 或 errors.As
|
||||
if errors.Is(err, ErrIndexNotFound) {
|
||||
// 处理索引不存在的情况
|
||||
}
|
||||
|
||||
// ❌ 错误:吞掉错误信息
|
||||
if err != nil {
|
||||
return errors.New("something went wrong")
|
||||
}
|
||||
```
|
||||
|
||||
**Defer 中的错误处理:**
|
||||
|
||||
对于 `defer` 语句中的 Close() 等清理操作,**不能**完全忽略错误:
|
||||
|
||||
```go
|
||||
// ✅ 正确:检查并记录 Close 错误
|
||||
defer func() {
|
||||
if closeErr := res.Body.Close(); closeErr != nil {
|
||||
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
|
||||
}
|
||||
}()
|
||||
|
||||
// ❌ 错误:完全忽略错误(虽然能通过 linter,但不是最佳实践)
|
||||
defer func() { _ = res.Body.Close() }()
|
||||
|
||||
// ❌ 错误:不处理错误(errcheck 警告)
|
||||
defer res.Body.Close()
|
||||
```
|
||||
|
||||
**原因说明:**
|
||||
- HTTP Response Body 的 Close() 错误虽然通常不影响业务逻辑(数据已读取)
|
||||
- 但完全忽略错误不符合 Go 最佳实践
|
||||
- 检查并记录到 stderr 便于调试和监控
|
||||
- 这种方式既满足 linter 要求,又提供诊断信息
|
||||
```
|
||||
|
||||
### 4. 注释规范
|
||||
|
||||
```go
|
||||
// ✅ 正确:包注释(在 package 语句之前)
|
||||
// Package client provides a high-level interface for interacting with AWS OpenSearch Service.
|
||||
// It handles connection management, authentication, and request signing.
|
||||
package client
|
||||
|
||||
// ✅ 正确:导出类型的注释
|
||||
// Client represents an OpenSearch client that manages connections and operations.
|
||||
type Client struct {
|
||||
// ...
|
||||
}
|
||||
|
||||
// ✅ 正确:导出方法的注释
|
||||
// Search executes a search query against the specified index.
|
||||
// It returns the search results or an error if the operation fails.
|
||||
func (c *Client) Search(ctx context.Context, indexName string, query Query) (*SearchResult, error) {
|
||||
// ...
|
||||
}
|
||||
|
||||
// ✅ 正确:复杂逻辑的内部注释
|
||||
func processResults(data []byte) error {
|
||||
// Parse JSON response
|
||||
var resp Response
|
||||
if err := json.Unmarshal(data, &resp); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Validate response structure
|
||||
if resp.Total == 0 {
|
||||
return ErrNoResults
|
||||
}
|
||||
|
||||
// Transform results
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### 5. 代码结构规范
|
||||
|
||||
**文件组织:**
|
||||
```go
|
||||
// 1. 包声明和包注释
|
||||
// Package client provides...
|
||||
package client
|
||||
|
||||
// 2. 导入语句(分组:标准库、第三方库、本地包)
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/opensearch-project/opensearch-go/v2"
|
||||
|
||||
"es-demo/config"
|
||||
"es-demo/models"
|
||||
)
|
||||
|
||||
// 3. 常量
|
||||
const (
|
||||
DefaultTimeout = 30 * time.Second
|
||||
)
|
||||
|
||||
// 4. 变量
|
||||
var (
|
||||
ErrInvalidConfig = errors.New("invalid configuration")
|
||||
)
|
||||
|
||||
// 5. 类型定义
|
||||
type Client struct {
|
||||
// ...
|
||||
}
|
||||
|
||||
// 6. 构造函数
|
||||
func NewClient(cfg *Config) (*Client, error) {
|
||||
// ...
|
||||
}
|
||||
|
||||
// 7. 方法(接收者方法)
|
||||
func (c *Client) Connect(ctx context.Context) error {
|
||||
// ...
|
||||
}
|
||||
|
||||
// 8. 其他函数
|
||||
func validateConfig(cfg *Config) error {
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
**接口设计:**
|
||||
```go
|
||||
// ✅ 正确:接口定义在使用方,而非实现方
|
||||
// 在 operations/search 包中定义
|
||||
type DocumentStore interface {
|
||||
GetDocument(ctx context.Context, id string) (*Document, error)
|
||||
PutDocument(ctx context.Context, doc *Document) error
|
||||
}
|
||||
|
||||
// ✅ 正确:保持接口小而专注
|
||||
type Searcher interface {
|
||||
Search(ctx context.Context, query Query) (*Result, error)
|
||||
}
|
||||
|
||||
type Indexer interface {
|
||||
CreateIndex(ctx context.Context, name string, settings map[string]any) error
|
||||
DeleteIndex(ctx context.Context, name string) error
|
||||
}
|
||||
```
|
||||
|
||||
### 6. 测试规范
|
||||
|
||||
**文件命名:**
|
||||
- 测试文件:`client_test.go`(对应 `client.go`)
|
||||
- 测试函数:`TestNewClient`、`TestClient_Search`
|
||||
|
||||
**测试结构:**
|
||||
```go
|
||||
func TestNewClient(t *testing.T) {
|
||||
// 使用表驱动测试
|
||||
tests := []struct {
|
||||
name string
|
||||
config *Config
|
||||
want bool // 期望成功/失败
|
||||
wantErr error
|
||||
}{
|
||||
{
|
||||
name: "valid config",
|
||||
config: &Config{
|
||||
Endpoint: "https://example.com",
|
||||
},
|
||||
want: true,
|
||||
wantErr: nil,
|
||||
},
|
||||
{
|
||||
name: "nil config",
|
||||
config: nil,
|
||||
want: false,
|
||||
wantErr: ErrInvalidConfig,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got, err := NewClient(tt.config)
|
||||
if !errors.Is(err, tt.wantErr) {
|
||||
t.Errorf("NewClient() error = %v, wantErr %v", err, tt.wantErr)
|
||||
return
|
||||
}
|
||||
if (got != nil) != tt.want {
|
||||
t.Errorf("NewClient() got = %v, want %v", got != nil, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 集成测试(需要真实环境)
|
||||
func TestClient_Search_Integration(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("skipping integration test in short mode")
|
||||
}
|
||||
|
||||
// 测试逻辑...
|
||||
}
|
||||
```
|
||||
|
||||
**辅助函数:**
|
||||
```go
|
||||
// 测试辅助函数以小写字母开头(不导出)
|
||||
func setupTestClient(t *testing.T) *Client {
|
||||
t.Helper()
|
||||
|
||||
cfg := &Config{
|
||||
Endpoint: "http://localhost:9200",
|
||||
}
|
||||
client, err := NewClient(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to create test client: %v", err)
|
||||
}
|
||||
return client
|
||||
}
|
||||
|
||||
func cleanupTestIndex(t *testing.T, client *Client, indexName string) {
|
||||
t.Helper()
|
||||
|
||||
if err := client.DeleteIndex(context.Background(), indexName); err != nil {
|
||||
t.Logf("cleanup failed: %v", err)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 项目阶段考虑
|
||||
|
||||
### 当前阶段(实验期)
|
||||
|
||||
**允许的简化:**
|
||||
|
||||
1. **配置管理**:可以使用全局变量
|
||||
```go
|
||||
// config/config.go
|
||||
package config
|
||||
|
||||
var (
|
||||
// Global configuration (acceptable in experimental phase)
|
||||
Endpoint string
|
||||
AccessKey string
|
||||
SecretKey string
|
||||
Region string
|
||||
)
|
||||
|
||||
func Init() {
|
||||
// Simple initialization from environment variables
|
||||
Endpoint = os.Getenv("ES_ENDPOINT")
|
||||
AccessKey = os.Getenv("AWS_ACCESS_KEY_ID")
|
||||
SecretKey = os.Getenv("AWS_SECRET_ACCESS_KEY")
|
||||
Region = os.Getenv("AWS_REGION")
|
||||
}
|
||||
```
|
||||
|
||||
2. **日志系统**:基础的 log 包使用
|
||||
```go
|
||||
// utils/logger.go
|
||||
package utils
|
||||
|
||||
import "log"
|
||||
|
||||
var (
|
||||
// Global logger (acceptable in experimental phase)
|
||||
Logger = log.New(os.Stdout, "[ES-Demo] ", log.LstdFlags)
|
||||
)
|
||||
|
||||
func Info(msg string, args ...any) {
|
||||
Logger.Printf("INFO: "+msg, args...)
|
||||
}
|
||||
|
||||
func Error(msg string, args ...any) {
|
||||
Logger.Printf("ERROR: "+msg, args...)
|
||||
}
|
||||
```
|
||||
|
||||
**但必须保持:**
|
||||
- 代码的可读性和可维护性
|
||||
- 清晰的职责划分
|
||||
- 完整的错误处理
|
||||
- 充分的注释
|
||||
|
||||
## 代码质量检查
|
||||
|
||||
### 使用 Linter 进行自动化检查
|
||||
|
||||
本项目使用 `golangci-lint` 进行代码质量检查。在生成或修改代码后,**必须**确保代码通过 linter 检查。
|
||||
|
||||
**配置文件:** `.golangci.yml`
|
||||
|
||||
**运行 Linter:**
|
||||
```bash
|
||||
# 检查所有代码
|
||||
golangci-lint run ./...
|
||||
|
||||
# 自动修复部分问题
|
||||
golangci-lint run --fix ./...
|
||||
```
|
||||
|
||||
**处理 Linter 输出:**
|
||||
1. 认真阅读每一条 linter 警告和错误
|
||||
2. 理解问题的根本原因
|
||||
3. 按照 Go 最佳实践修复问题
|
||||
4. 不要通过禁用 linter 规则来"修复"问题(除非有充分理由)
|
||||
5. 如需禁用某个规则,必须在配置文件中注释说明原因
|
||||
|
||||
**常见 Linter 问题及解决方案:**
|
||||
|
||||
- `errcheck`: 错误未处理 → 添加错误处理逻辑
|
||||
- `gofmt`: 格式不正确 → 运行 `go fmt ./...`
|
||||
- `govet`: 可疑构造 → 重构代码
|
||||
- `ineffassign`: 无效赋值 → 移除或修正赋值
|
||||
- `staticcheck`: 静态分析问题 → 按建议修改
|
||||
|
||||
**集成到开发流程:**
|
||||
- 在提交代码前,运行测试流水线(包含 linter 检查)
|
||||
- 确保 linter 零警告、零错误
|
||||
- CI/CD 流水线应包含 linter 检查步骤
|
||||
|
||||
## 常见模式和示例
|
||||
|
||||
### 客户端初始化模式
|
||||
```go
|
||||
// Config holds the configuration for the OpenSearch client.
|
||||
type Config struct {
|
||||
Endpoint string
|
||||
Region string
|
||||
AccessKey string
|
||||
SecretKey string
|
||||
Timeout time.Duration
|
||||
}
|
||||
|
||||
// Client represents an OpenSearch client.
|
||||
type Client struct {
|
||||
config *Config
|
||||
client *opensearch.Client
|
||||
}
|
||||
|
||||
// NewClient creates a new OpenSearch client with the given configuration.
|
||||
func NewClient(cfg *Config) (*Client, error) {
|
||||
if cfg == nil {
|
||||
return nil, fmt.Errorf("config cannot be nil")
|
||||
}
|
||||
|
||||
if err := validateConfig(cfg); err != nil {
|
||||
return nil, fmt.Errorf("invalid config: %w", err)
|
||||
}
|
||||
|
||||
// Initialize OpenSearch client
|
||||
osClient, err := opensearch.NewClient(opensearch.Config{
|
||||
Addresses: []string{cfg.Endpoint},
|
||||
// ... other settings
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create opensearch client: %w", err)
|
||||
}
|
||||
|
||||
return &Client{
|
||||
config: cfg,
|
||||
client: osClient,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func validateConfig(cfg *Config) error {
|
||||
if cfg.Endpoint == "" {
|
||||
return errors.New("endpoint is required")
|
||||
}
|
||||
if cfg.Region == "" {
|
||||
return errors.New("region is required")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### 操作方法模式
|
||||
```go
|
||||
// SearchRequest represents a search query request.
|
||||
type SearchRequest struct {
|
||||
Index string
|
||||
Query map[string]any
|
||||
Size int
|
||||
From int
|
||||
}
|
||||
|
||||
// SearchResult represents the response from a search operation.
|
||||
type SearchResult struct {
|
||||
Hits []Document
|
||||
Total int64
|
||||
Took int64
|
||||
}
|
||||
|
||||
// Search executes a search query against the specified index.
|
||||
func (c *Client) Search(ctx context.Context, req *SearchRequest) (*SearchResult, error) {
|
||||
if req == nil {
|
||||
return nil, errors.New("search request cannot be nil")
|
||||
}
|
||||
|
||||
if req.Index == "" {
|
||||
return nil, errors.New("index name is required")
|
||||
}
|
||||
|
||||
// Build and execute search
|
||||
// ...
|
||||
|
||||
return result, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Context 使用模式
|
||||
```go
|
||||
// 所有可能阻塞或需要超时控制的操作都应接受 context
|
||||
func (c *Client) CreateIndex(ctx context.Context, name string, settings map[string]any) error {
|
||||
// 使用 context 控制超时
|
||||
req, err := http.NewRequestWithContext(ctx, "PUT", c.buildURL(name), body)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create request: %w", err)
|
||||
}
|
||||
|
||||
// ...
|
||||
}
|
||||
|
||||
// 调用示例
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
|
||||
err := client.CreateIndex(ctx, "my-index", settings)
|
||||
```
|
||||
|
||||
## 版本和依赖管理
|
||||
|
||||
**依赖选择原则:**
|
||||
1. 优先使用最新稳定版本
|
||||
2. 使用官方推荐的库(如 opensearch-go)
|
||||
3. 定期更新依赖(使用 `go get -u` 和 `go mod tidy`)
|
||||
|
||||
**更新依赖时检查:**
|
||||
```bash
|
||||
# 查看可用更新
|
||||
go list -u -m all
|
||||
|
||||
# 更新特定依赖
|
||||
go get -u github.com/opensearch-project/opensearch-go/v2@latest
|
||||
|
||||
# 整理依赖
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
## 测试流水线
|
||||
|
||||
### 流水线脚本要求
|
||||
|
||||
项目**必须**包含自动化测试流水线脚本,用于执行完整的代码质量检查流程。
|
||||
|
||||
**脚本命名规范:**
|
||||
- Windows (PowerShell): `test.ps1`
|
||||
- Linux/Mac (Bash): `test.sh`
|
||||
|
||||
**流水线阶段(按顺序执行):**
|
||||
|
||||
1. **Linting** - 代码质量检查
|
||||
```bash
|
||||
golangci-lint run ./...
|
||||
```
|
||||
|
||||
2. **Build** - 编译验证
|
||||
```bash
|
||||
# 编译但不保留二进制文件
|
||||
go build -o /dev/null . # Linux/Mac
|
||||
go build -o NUL . # Windows
|
||||
```
|
||||
|
||||
3. **Test** - 运行测试
|
||||
```bash
|
||||
go test -v -short -coverprofile=coverage.out ./...
|
||||
```
|
||||
|
||||
4. **Cleanup** - 清理构建产物
|
||||
```bash
|
||||
# 删除编译输出、覆盖率文件等
|
||||
rm -f *.exe *.test coverage.out coverage.html
|
||||
```
|
||||
|
||||
### 流水线脚本检查
|
||||
|
||||
**在最终测试前,AI Agent 必须检查:**
|
||||
|
||||
1. 当前平台是否存在对应的流水线脚本
|
||||
2. 如果不存在但有其他平台的脚本,则根据当前平台创建对应版本
|
||||
|
||||
**平台检测和脚本生成逻辑:**
|
||||
|
||||
```go
|
||||
// 伪代码示例
|
||||
if platform == "Windows" {
|
||||
if !exists("test.ps1") {
|
||||
if exists("test.sh") {
|
||||
// 将 test.sh 翻译为 test.ps1
|
||||
createWindowsScript()
|
||||
} else {
|
||||
// 创建新的 Windows 脚本
|
||||
createDefaultWindowsScript()
|
||||
}
|
||||
}
|
||||
} else if platform == "Linux" || platform == "Mac" {
|
||||
if !exists("test.sh") {
|
||||
if exists("test.ps1") {
|
||||
// 将 test.ps1 翻译为 test.sh
|
||||
createUnixScript()
|
||||
} else {
|
||||
// 创建新的 Unix 脚本
|
||||
createDefaultUnixScript()
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### PowerShell 脚本示例 (test.ps1)
|
||||
|
||||
```powershell
|
||||
#!/usr/bin/env pwsh
|
||||
# 测试流水线脚本 - Windows PowerShell
|
||||
|
||||
param(
|
||||
[switch]$SkipLint,
|
||||
[switch]$SkipBuild,
|
||||
[switch]$SkipTest,
|
||||
[switch]$Verbose
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
# 1. Linting
|
||||
if (-not $SkipLint) {
|
||||
Write-Host "=== Linting ===" -ForegroundColor Yellow
|
||||
golangci-lint run ./...
|
||||
if ($LASTEXITCODE -ne 0) { exit 1 }
|
||||
}
|
||||
|
||||
# 2. Build
|
||||
if (-not $SkipBuild) {
|
||||
Write-Host "=== Build ===" -ForegroundColor Yellow
|
||||
go build -o NUL .
|
||||
if ($LASTEXITCODE -ne 0) { exit 1 }
|
||||
}
|
||||
|
||||
# 3. Test
|
||||
if (-not $SkipTest) {
|
||||
Write-Host "=== Test ===" -ForegroundColor Yellow
|
||||
go test -v -short -coverprofile=coverage.out ./...
|
||||
if ($LASTEXITCODE -ne 0) { exit 1 }
|
||||
go tool cover -func=coverage.out | Select-String "total:"
|
||||
}
|
||||
|
||||
# 4. Cleanup
|
||||
Write-Host "=== Cleanup ===" -ForegroundColor Yellow
|
||||
Remove-Item -Path "*.exe","*.test","coverage.out","coverage.html" -ErrorAction SilentlyContinue
|
||||
|
||||
Write-Host "✅ Pipeline PASSED" -ForegroundColor Green
|
||||
```
|
||||
|
||||
### Bash 脚本示例 (test.sh)
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
# 测试流水线脚本 - Linux/Mac Bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SKIP_LINT=false
|
||||
SKIP_BUILD=false
|
||||
SKIP_TEST=false
|
||||
VERBOSE=false
|
||||
|
||||
# 解析参数
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case $1 in
|
||||
--skip-lint) SKIP_LINT=true; shift ;;
|
||||
--skip-build) SKIP_BUILD=true; shift ;;
|
||||
--skip-test) SKIP_TEST=true; shift ;;
|
||||
--verbose) VERBOSE=true; shift ;;
|
||||
*) echo "Unknown option: $1"; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# 1. Linting
|
||||
if [ "$SKIP_LINT" = false ]; then
|
||||
echo "=== Linting ==="
|
||||
golangci-lint run ./...
|
||||
fi
|
||||
|
||||
# 2. Build
|
||||
if [ "$SKIP_BUILD" = false ]; then
|
||||
echo "=== Build ==="
|
||||
go build -o /dev/null .
|
||||
fi
|
||||
|
||||
# 3. Test
|
||||
if [ "$SKIP_TEST" = false ]; then
|
||||
echo "=== Test ==="
|
||||
go test -v -short -race -coverprofile=coverage.out ./...
|
||||
go tool cover -func=coverage.out | grep total:
|
||||
fi
|
||||
|
||||
# 4. Cleanup
|
||||
echo "=== Cleanup ==="
|
||||
rm -f *.exe *.test coverage.out coverage.html
|
||||
|
||||
echo "✅ Pipeline PASSED"
|
||||
```
|
||||
|
||||
### 使用流水线
|
||||
|
||||
**日常开发:**
|
||||
```bash
|
||||
# Windows
|
||||
powershell -ExecutionPolicy Bypass -File .\test.ps1
|
||||
|
||||
# Linux/Mac
|
||||
chmod +x test.sh
|
||||
./test.sh
|
||||
```
|
||||
|
||||
**跳过特定阶段:**
|
||||
```bash
|
||||
# Windows - 跳过 linting
|
||||
powershell -ExecutionPolicy Bypass -File .\test.ps1 -SkipLint
|
||||
|
||||
# Linux/Mac - 跳过 build
|
||||
./test.sh --skip-build
|
||||
```
|
||||
|
||||
**CI/CD 集成:**
|
||||
```yaml
|
||||
# GitHub Actions 示例
|
||||
- name: Run test pipeline
|
||||
run: |
|
||||
if [ "$RUNNER_OS" == "Windows" ]; then
|
||||
pwsh -File test.ps1
|
||||
else
|
||||
./test.sh
|
||||
fi
|
||||
```
|
||||
|
||||
### 流水线最佳实践
|
||||
|
||||
1. **构建产物管理:**
|
||||
- 编译时不保留二进制文件(输出到 `/dev/null` 或 `NUL`)
|
||||
- 测试后自动清理所有临时文件
|
||||
- 在 `.gitignore` 中排除所有构建产物
|
||||
|
||||
2. **失败快速原则:**
|
||||
- 任何阶段失败立即终止流水线
|
||||
- 返回非零退出码
|
||||
- 提供清晰的错误信息
|
||||
|
||||
3. **可配置性:**
|
||||
- 提供跳过特定阶段的选项
|
||||
- 支持详细输出模式
|
||||
- 允许自定义超时设置
|
||||
|
||||
4. **跨平台一致性:**
|
||||
- 确保 Windows 和 Linux/Mac 脚本功能一致
|
||||
- 使用等效的命令和选项
|
||||
- 测试结果应该相同
|
||||
|
||||
## 文档要求
|
||||
|
||||
**代码文档:**
|
||||
- 所有导出的标识符必须有文档注释
|
||||
- 复杂的内部逻辑添加解释性注释
|
||||
- 注释应解释"为什么"而不仅仅是"是什么"
|
||||
|
||||
**README 维护:**
|
||||
- 添加新功能时更新功能列表
|
||||
- 添加示例代码展示用法
|
||||
- 更新配置说明
|
||||
|
||||
**示例代码:**
|
||||
- 在 `examples/` 目录提供可运行的示例
|
||||
- 示例代码必须可编译和运行
|
||||
- 包含必要的错误处理
|
||||
|
||||
## AI Agent 工作流程(TDD 原则)
|
||||
|
||||
当收到开发任务时,**必须严格遵循 TDD(测试驱动开发)流程**:
|
||||
|
||||
### 1. Red(红灯阶段)- 编写失败的测试
|
||||
|
||||
**理解需求:**
|
||||
- 明确要实现的功能和预期行为
|
||||
- 确定输入、输出和边界条件
|
||||
- 识别可能的错误场景
|
||||
|
||||
**设计 API:**
|
||||
- 定义函数签名和类型
|
||||
- 设计清晰的接口
|
||||
- 确定错误返回值
|
||||
|
||||
**编写测试(先于实现):**
|
||||
```go
|
||||
// 示例:先写测试
|
||||
func TestNewClient(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
config *Config
|
||||
wantErr error
|
||||
}{
|
||||
{
|
||||
name: "valid config",
|
||||
config: &Config{Endpoint: "https://example.com"},
|
||||
wantErr: nil,
|
||||
},
|
||||
{
|
||||
name: "nil config",
|
||||
config: nil,
|
||||
wantErr: ErrInvalidConfig,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got, err := NewClient(tt.config)
|
||||
if !errors.Is(err, tt.wantErr) {
|
||||
t.Errorf("NewClient() error = %v, wantErr %v", err, tt.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**验证测试失败:**
|
||||
- 运行测试,确认测试失败(因为功能还未实现)
|
||||
- 确保测试失败的原因是正确的
|
||||
|
||||
### 2. Green(绿灯阶段)- 编写最小实现
|
||||
|
||||
**实现功能:**
|
||||
- 编写最少的代码让测试通过
|
||||
- 遵循本文档的所有规范
|
||||
- 添加完整的文档注释
|
||||
- 实现完整的错误处理
|
||||
|
||||
```go
|
||||
// 示例:实现功能
|
||||
func NewClient(cfg *Config) (*Client, error) {
|
||||
if cfg == nil {
|
||||
return nil, ErrInvalidConfig
|
||||
}
|
||||
|
||||
// 最小实现
|
||||
return &Client{config: cfg}, nil
|
||||
}
|
||||
```
|
||||
|
||||
**运行测试:**
|
||||
- 确保所有测试通过
|
||||
- 检查测试覆盖率
|
||||
|
||||
### 3. Refactor(重构阶段)- 优化代码
|
||||
|
||||
**代码重构:**
|
||||
- 消除重复代码
|
||||
- 改进命名和结构
|
||||
- 优化性能(如需要)
|
||||
- 确保代码可读性
|
||||
|
||||
**运行 Linter:**
|
||||
```bash
|
||||
golangci-lint run ./...
|
||||
```
|
||||
|
||||
**修复所有 Linter 问题**
|
||||
|
||||
**再次运行测试:**
|
||||
- 确保重构后测试仍然通过
|
||||
- 验证代码行为未改变
|
||||
|
||||
### 4. 验证和文档
|
||||
|
||||
**运行测试流水线:**
|
||||
```bash
|
||||
# Windows
|
||||
powershell -ExecutionPolicy Bypass -File .\test.ps1
|
||||
|
||||
# Linux/Mac
|
||||
./test.sh
|
||||
```
|
||||
|
||||
**更新文档:**
|
||||
- 更新模块 README(如需要)
|
||||
- 更新项目 README(如需要)
|
||||
- 添加使用示例(如需要)
|
||||
|
||||
### TDD 循环总结
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ 1. 编写失败的测试 (Red) │
|
||||
│ - 定义预期行为 │
|
||||
│ - 验证测试确实失败 │
|
||||
└──────────────┬──────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────┐
|
||||
│ 2. 编写最小实现 (Green) │
|
||||
│ - 让测试通过 │
|
||||
│ - 不过度设计 │
|
||||
└──────────────┬──────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────┐
|
||||
│ 3. 重构优化 (Refactor) │
|
||||
│ - 改进代码质量 │
|
||||
│ - 运行 Linter │
|
||||
│ - 确保测试仍然通过 │
|
||||
└──────────────┬──────────────────────┘
|
||||
↓
|
||||
┌─────────┐
|
||||
│ 下一功能 │
|
||||
└─────────┘
|
||||
```
|
||||
|
||||
**重要提醒:**
|
||||
- ❌ 不要先写实现再补测试
|
||||
- ✅ 必须先写测试,然后写实现
|
||||
- ✅ 每次只关注一个小功能
|
||||
- ✅ 频繁运行测试,保持快速反馈
|
||||
|
||||
## 特殊指令
|
||||
|
||||
### 生成新模块时
|
||||
1. 创建包目录和主文件
|
||||
2. 创建对应的测试文件
|
||||
3. 在 README 中更新项目结构
|
||||
4. 在功能列表中标记进度
|
||||
|
||||
### 重构代码时
|
||||
1. 确保向后兼容(除非明确要求破坏性变更)
|
||||
2. 更新所有相关测试
|
||||
3. 更新文档和注释
|
||||
4. 运行完整测试套件
|
||||
|
||||
### 修复 Bug 时
|
||||
1. 先编写复现 bug 的测试
|
||||
2. 修复代码
|
||||
3. 确保测试通过
|
||||
4. 添加注释说明修复内容
|
||||
|
||||
## 禁止事项
|
||||
|
||||
**绝对不允许:**
|
||||
- ❌ 忽略错误(使用 `_` 丢弃错误)
|
||||
- ❌ 导出的标识符没有注释
|
||||
- ❌ 使用 `panic` 处理常规错误(除非确实是不可恢复的错误)
|
||||
- ❌ 在库代码中使用 `os.Exit`
|
||||
- ❌ 硬编码配置值(应使用配置文件或环境变量)
|
||||
- ❌ 使用全局可变状态(除了当前阶段允许的配置和日志)
|
||||
|
||||
**应当避免:**
|
||||
- ⚠️ 过长的函数(超过 50 行考虑拆分)
|
||||
- ⚠️ 过深的嵌套(超过 3 层考虑重构)
|
||||
- ⚠️ 重复代码(提取为函数)
|
||||
- ⚠️ 神秘的数字和字符串(使用命名常量)
|
||||
|
||||
## 参考资源
|
||||
|
||||
- [Effective Go](https://go.dev/doc/effective_go)
|
||||
- [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments)
|
||||
- [Go 标准库文档](https://pkg.go.dev/std)
|
||||
- [OpenSearch Go Client](https://github.com/opensearch-project/opensearch-go)
|
||||
- [AWS SDK for Go v2](https://aws.github.io/aws-sdk-go-v2/)
|
||||
|
||||
---
|
||||
|
||||
**总结**: 作为 AI Agent,你的目标是生成生产级别的 Go 代码,遵循所有最佳实践,为将来转化为可复用库打下坚实基础。当有疑问时,参考 Go 标准库的实现方式。
|
||||
37
.gitignore
vendored
Normal file
37
.gitignore
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
# Binaries for programs and plugins
|
||||
*.exe
|
||||
*.exe~
|
||||
*.dll
|
||||
*.so
|
||||
*.dylib
|
||||
|
||||
# Test binary, built with `go test -c`
|
||||
*.test
|
||||
|
||||
# Output of the go coverage tool
|
||||
*.out
|
||||
coverage
|
||||
coverage.out
|
||||
coverage.html
|
||||
|
||||
# Dependency directories
|
||||
vendor/
|
||||
|
||||
# Go workspace file
|
||||
go.work
|
||||
|
||||
# Environment variables
|
||||
.env
|
||||
.env.local
|
||||
.env.test
|
||||
|
||||
# IDE specific files
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# OS specific files
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
67
.golangci.yml
Normal file
67
.golangci.yml
Normal file
@@ -0,0 +1,67 @@
|
||||
# golangci-lint 配置文件
|
||||
# 详细文档: https://golangci-lint.run/usage/configuration/
|
||||
|
||||
run:
|
||||
timeout: 5m
|
||||
tests: true
|
||||
modules-download-mode: readonly
|
||||
|
||||
output:
|
||||
formats:
|
||||
- format: colored-line-number
|
||||
print-issued-lines: true
|
||||
print-linter-name: true
|
||||
sort-results: true
|
||||
|
||||
linters:
|
||||
enable:
|
||||
# 默认启用的 linters
|
||||
- gosimple # 简化代码
|
||||
- ineffassign # 检查无效赋值
|
||||
- staticcheck # 静态检查
|
||||
- unused # 检查未使用的代码
|
||||
|
||||
# 额外启用的 linters
|
||||
- gofmt # 检查格式化
|
||||
- goimports # 检查导入顺序
|
||||
- misspell # 检查拼写错误
|
||||
- gosec # 安全检查
|
||||
- unconvert # 检查不必要的类型转换
|
||||
- gocyclo # 检查代码复杂度
|
||||
- goconst # 检查可以提取为常量的字符串
|
||||
- whitespace # 检查空白符
|
||||
|
||||
linters-settings:
|
||||
gocyclo:
|
||||
min-complexity: 15
|
||||
|
||||
gosec:
|
||||
excludes:
|
||||
- G404 # 允许使用 math/rand
|
||||
|
||||
gofmt:
|
||||
simplify: true
|
||||
|
||||
goimports:
|
||||
local-prefixes: es-demo
|
||||
|
||||
issues:
|
||||
exclude-dirs:
|
||||
- vendor
|
||||
exclude-use-default: false
|
||||
max-issues-per-linter: 0
|
||||
max-same-issues: 0
|
||||
|
||||
exclude-rules:
|
||||
# 测试文件中允许一些宽松的规则
|
||||
- path: _test\.go
|
||||
linters:
|
||||
- errcheck
|
||||
- gosec
|
||||
- gocyclo
|
||||
- goconst
|
||||
|
||||
# 允许在 main 包中使用 fmt.Printf
|
||||
- path: main\.go
|
||||
linters:
|
||||
- forbidigo
|
||||
256
README.md
Normal file
256
README.md
Normal file
@@ -0,0 +1,256 @@
|
||||
# ES-Demo
|
||||
|
||||
OpenSearch/Elasticsearch 客户端实验项目,用于测试和验证 AWS OpenSearch Service 的各种功能和接口。
|
||||
|
||||
## 项目概述
|
||||
|
||||
这是一个用于实验 AWS OpenSearch Service 的 Go 项目。项目的主要目标是:
|
||||
|
||||
- 测试 OpenSearch 客户端的各种接口和功能
|
||||
- 实现生产级别的代码质量
|
||||
- 保持代码结构清晰、模块化
|
||||
- 完备的测试覆盖
|
||||
- 为未来转化为可复用的依赖库做准备
|
||||
|
||||
## 项目特点
|
||||
|
||||
- **生产级代码质量**:所有代码遵循 Go 官方最佳实践
|
||||
- **模块化设计**:清晰的目录结构和职责划分
|
||||
- **AI 辅助开发**:大部分代码由 AI 辅助生成,配合详细的指令文件
|
||||
- **测试完备**:包含单元测试和集成测试
|
||||
- **最新依赖**:使用最新版本的依赖库(除非有特殊需求)
|
||||
|
||||
## 技术栈
|
||||
|
||||
- **语言**: Go 1.25.3
|
||||
- **OpenSearch 客户端**: opensearch-go/v2 v2.3.0
|
||||
- **AWS SDK**: aws-sdk-go-v2 (config, aws/signer/v4)
|
||||
- **环境变量**: godotenv v1.5.1
|
||||
- **代码质量**: golangci-lint v1.64.8
|
||||
- **测试框架**: Go 标准库 testing
|
||||
|
||||
## 项目结构
|
||||
|
||||
```text
|
||||
es-demo/
|
||||
├── README.md # 项目说明文档
|
||||
├── .env.example # 环境变量配置示例
|
||||
├── .gitignore # Git 忽略文件配置
|
||||
├── .golangci.yml # golangci-lint 配置
|
||||
├── test.ps1 # 测试流水线脚本(Windows)
|
||||
├── go.mod # Go 模块依赖
|
||||
├── go.sum # 依赖校验和
|
||||
├── main.go # 程序入口
|
||||
├── .github/
|
||||
│ └── instructions/ # AI Agent 开发指令
|
||||
│ ├── CLAUDE_GUIDELINE.instructions.md
|
||||
│ └── DEV_GUIDELINE.instructions.md
|
||||
├── config/ # 配置相关
|
||||
│ ├── config.go # 全局配置,支持 .env 文件加载
|
||||
│ └── config_test.go # 配置测试(100% 覆盖率)
|
||||
├── client/ # OpenSearch 客户端封装
|
||||
│ ├── client.go # 客户端连接管理和 AWS 认证
|
||||
│ └── client_test.go # 客户端测试
|
||||
└── operations/ # 业务操作实现
|
||||
├── cluster/ # 集群级别运维操作
|
||||
│ ├── cluster.go # 集群信息查询
|
||||
│ └── cluster_test.go # 集群操作测试
|
||||
└── index/ # 索引级别业务操作
|
||||
├── template.go # 索引模板管理(CRUD)
|
||||
└── template_test.go # 模板操作测试
|
||||
```
|
||||
|
||||
## 开发原则
|
||||
|
||||
### 代码质量要求
|
||||
|
||||
1. **遵循 Go 官方最佳实践**
|
||||
- 使用 `gofmt` 格式化代码
|
||||
- 遵循 [Effective Go](https://go.dev/doc/effective_go) 指南
|
||||
- 遵循 [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments)
|
||||
- 使用 `go vet` 进行静态检查
|
||||
- 使用 `golangci-lint` 进行代码检查
|
||||
|
||||
2. **命名规范**
|
||||
- 包名:简短、小写、单数形式
|
||||
- 接口名:单个方法接口以 `-er` 结尾
|
||||
- 变量名:驼峰命名,缩写词全大写(如 `URL`、`ID`)
|
||||
- 常量:驼峰命名或全大写下划线分隔
|
||||
|
||||
3. **错误处理**
|
||||
- 所有错误必须处理
|
||||
- 使用有意义的错误信息
|
||||
- 适当时使用 `errors.Is()` 和 `errors.As()`
|
||||
- 可考虑使用 `fmt.Errorf()` 包装错误
|
||||
|
||||
4. **注释规范**
|
||||
- 所有导出的类型、函数必须有注释
|
||||
- 注释以类型或函数名开头
|
||||
- 包注释放在 `package` 语句之前
|
||||
|
||||
### 非功能需求(起步阶段)
|
||||
|
||||
由于项目处于实验阶段,以下方面采用简化实现:
|
||||
|
||||
- **配置管理**:使用全局变量,简单的配置结构
|
||||
- **日志系统**:基础的日志输出,可使用全局 logger
|
||||
- **错误处理**:基本的错误返回和处理,暂不需要复杂的错误链
|
||||
- **性能优化**:功能优先,性能问题后续优化
|
||||
|
||||
这些简化不影响代码的可读性和可维护性,为后续改进预留空间。
|
||||
|
||||
## 功能模块
|
||||
|
||||
### 已完成功能
|
||||
|
||||
#### 客户端连接 (client/)
|
||||
|
||||
- [x] OpenSearch 客户端初始化和连接管理
|
||||
- [x] AWS v4 签名认证
|
||||
- [x] 配置验证和错误处理
|
||||
- [x] 支持 .env 文件配置加载
|
||||
|
||||
#### 集群运维操作 (operations/cluster/)
|
||||
|
||||
- [x] 获取集群信息(GetInfo)
|
||||
- 集群名称、版本、UUID
|
||||
- 节点数、分片数统计
|
||||
|
||||
#### 索引业务操作 (operations/index/)
|
||||
|
||||
- [x] 索引模板管理(Index Templates)
|
||||
- PutTemplate: 创建/更新索引模板
|
||||
- GetTemplate: 获取指定模板配置
|
||||
- DeleteTemplate: 删除索引模板
|
||||
- ListTemplates: 列出所有模板
|
||||
- 模板配置验证
|
||||
|
||||
### 待实现功能
|
||||
|
||||
以下功能将根据实际需求逐步实现:
|
||||
|
||||
- [ ] 索引管理(创建、删除、更新索引)
|
||||
- [ ] 文档操作(CRUD)
|
||||
- [ ] 搜索功能(基础查询、复杂查询)
|
||||
- [ ] 聚合查询
|
||||
- [ ] 批量操作
|
||||
- [ ] 性能测试工具
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 环境要求
|
||||
|
||||
- Go 1.25.3 或更高版本
|
||||
- AWS 账号和 OpenSearch Service 实例
|
||||
- AWS 访问凭证(Access Key ID 和 Secret Access Key)
|
||||
|
||||
### 安装依赖
|
||||
|
||||
```bash
|
||||
go mod download
|
||||
```
|
||||
|
||||
### 配置
|
||||
|
||||
[配置说明待补充]
|
||||
|
||||
### 运行
|
||||
|
||||
```bash
|
||||
go run main.go
|
||||
```
|
||||
|
||||
### 测试
|
||||
|
||||
#### 测试流水线(推荐)
|
||||
|
||||
使用自动化测试流水线,包含 Linting、Build、Test 和 Cleanup 四个阶段:
|
||||
|
||||
```powershell
|
||||
# Windows PowerShell
|
||||
powershell -ExecutionPolicy Bypass -File .\test.ps1
|
||||
|
||||
# 跳过特定阶段
|
||||
powershell -ExecutionPolicy Bypass -File .\test.ps1 -SkipLint
|
||||
powershell -ExecutionPolicy Bypass -File .\test.ps1 -SkipBuild
|
||||
```
|
||||
|
||||
#### 手动测试
|
||||
|
||||
```bash
|
||||
# 代码质量检查
|
||||
golangci-lint run ./...
|
||||
|
||||
# 编译检查
|
||||
go build -o NUL .
|
||||
|
||||
# 运行所有测试(跳过集成测试)
|
||||
go test -v -short ./...
|
||||
|
||||
# 运行测试并显示覆盖率
|
||||
go test -v -short -coverprofile=coverage.out ./...
|
||||
go tool cover -func=coverage.out
|
||||
|
||||
# 生成 HTML 覆盖率报告
|
||||
go tool cover -html=coverage.out -o coverage.html
|
||||
```
|
||||
|
||||
**注意**: 使用 `-short` 标志跳过需要真实 OpenSearch 实例的集成测试。
|
||||
|
||||
## AI 辅助开发
|
||||
|
||||
本项目大量使用 AI 辅助开发,遵循严格的 TDD(测试驱动开发)流程。详细的开发指令请参考:
|
||||
|
||||
- [开发准则](./.github/instructions/DEV_GUIDELINE.instructions.md) - Go 最佳实践、TDD 流程、测试流水线要求
|
||||
- [Agent 执行协议](./.github/instructions/CLAUDE_GUIDELINE.instructions.md) - Agent 工作流程和执行约束
|
||||
|
||||
## 开发路线图
|
||||
|
||||
### 第一阶段:基础设施(已完成)
|
||||
|
||||
- [x] 项目初始化
|
||||
- [x] 基础项目结构(Client + Operations 架构)
|
||||
- [x] OpenSearch 客户端封装(连接管理、AWS 认证)
|
||||
- [x] 基本测试框架(单元测试、集成测试分离)
|
||||
- [x] 测试流水线自动化(Lint → Build → Test → Cleanup)
|
||||
- [x] 代码质量保障(golangci-lint 集成)
|
||||
|
||||
### 第二阶段:核心功能(进行中)
|
||||
|
||||
- [x] 集群运维操作(GetInfo)
|
||||
- [x] 索引模板管理(CRUD)
|
||||
- [ ] 索引管理功能(创建、删除、更新索引)
|
||||
- [ ] 文档 CRUD 操作
|
||||
- [ ] 基础搜索功能
|
||||
- [ ] 完善测试覆盖
|
||||
|
||||
### 第三阶段:高级功能
|
||||
|
||||
- [ ] 聚合查询
|
||||
- [ ] 批量操作优化
|
||||
- [ ] 性能测试工具
|
||||
- [ ] 错误处理优化
|
||||
- [ ] 重试和容错机制
|
||||
|
||||
### 第四阶段:库化准备
|
||||
|
||||
- [ ] API 设计优化
|
||||
- [ ] 完整文档
|
||||
- [ ] 示例代码
|
||||
- [ ] 发布为可复用库
|
||||
|
||||
## 贡献指南
|
||||
|
||||
由于本项目主要用于个人实验,暂不接受外部贡献。但欢迎提出建议和问题。
|
||||
|
||||
## 许可证
|
||||
|
||||
[待定]
|
||||
|
||||
## 联系方式
|
||||
|
||||
[待补充]
|
||||
|
||||
---
|
||||
|
||||
**注意**: 本项目处于积极开发中,API 可能会发生变化。
|
||||
135
client/client.go
Normal file
135
client/client.go
Normal file
@@ -0,0 +1,135 @@
|
||||
// Package client provides a high-level interface for interacting with AWS OpenSearch Service.
|
||||
// It handles connection management, authentication, and request signing.
|
||||
package client
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/aws/aws-sdk-go-v2/aws"
|
||||
v4 "github.com/aws/aws-sdk-go-v2/aws/signer/v4"
|
||||
"github.com/opensearch-project/opensearch-go/v2"
|
||||
requestsigner "github.com/opensearch-project/opensearch-go/v2/signer/awsv2"
|
||||
)
|
||||
|
||||
var (
|
||||
// ErrInvalidConfig is returned when the client configuration is invalid.
|
||||
ErrInvalidConfig = errors.New("invalid configuration")
|
||||
|
||||
// ErrRequestFailed is returned when a request to OpenSearch fails.
|
||||
ErrRequestFailed = errors.New("request failed")
|
||||
)
|
||||
|
||||
const (
|
||||
// DefaultTimeout is the default timeout for requests.
|
||||
DefaultTimeout = 30 * time.Second
|
||||
)
|
||||
|
||||
// Config holds the configuration for the OpenSearch client.
|
||||
type Config struct {
|
||||
// Endpoint is the OpenSearch cluster endpoint URL.
|
||||
Endpoint string
|
||||
|
||||
// Region is the AWS region where the OpenSearch cluster is located.
|
||||
Region string
|
||||
|
||||
// AccessKey is the AWS access key ID for authentication.
|
||||
AccessKey string
|
||||
|
||||
// SecretKey is the AWS secret access key for authentication.
|
||||
SecretKey string
|
||||
|
||||
// Timeout is the request timeout duration. Defaults to DefaultTimeout if not set.
|
||||
Timeout time.Duration
|
||||
}
|
||||
|
||||
// Client represents an OpenSearch client that manages connections.
|
||||
type Client struct {
|
||||
config *Config
|
||||
client *opensearch.Client
|
||||
signer *v4.Signer
|
||||
}
|
||||
|
||||
// NewClient creates a new OpenSearch client with the given configuration.
|
||||
// It returns an error if the configuration is invalid or if the client cannot be initialized.
|
||||
func NewClient(cfg *Config) (*Client, error) {
|
||||
if cfg == nil {
|
||||
return nil, fmt.Errorf("%w: config cannot be nil", ErrInvalidConfig)
|
||||
}
|
||||
|
||||
if err := validateConfig(cfg); err != nil {
|
||||
return nil, fmt.Errorf("%w: %v", ErrInvalidConfig, err)
|
||||
}
|
||||
|
||||
// TODO: Use cfg.Timeout for client configuration
|
||||
// Currently timeout is not directly supported by opensearch-go client config
|
||||
|
||||
// Create AWS credentials
|
||||
awsConfig := aws.Config{
|
||||
Region: cfg.Region,
|
||||
Credentials: aws.CredentialsProviderFunc(func(ctx context.Context) (aws.Credentials, error) {
|
||||
return aws.Credentials{
|
||||
AccessKeyID: cfg.AccessKey,
|
||||
SecretAccessKey: cfg.SecretKey,
|
||||
}, nil
|
||||
}),
|
||||
}
|
||||
|
||||
// Create AWS v4 signer
|
||||
signer := v4.NewSigner()
|
||||
|
||||
// Create request signer for OpenSearch
|
||||
awsSigner, err := requestsigner.NewSignerWithService(awsConfig, "es")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create request signer: %w", err)
|
||||
}
|
||||
|
||||
// Configure OpenSearch client
|
||||
osConfig := opensearch.Config{
|
||||
Addresses: []string{cfg.Endpoint},
|
||||
Transport: &http.Transport{
|
||||
TLSClientConfig: &tls.Config{
|
||||
MinVersion: tls.VersionTLS12,
|
||||
},
|
||||
},
|
||||
Signer: awsSigner,
|
||||
}
|
||||
|
||||
// Create OpenSearch client
|
||||
osClient, err := opensearch.NewClient(osConfig)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create opensearch client: %w", err)
|
||||
}
|
||||
|
||||
return &Client{
|
||||
config: cfg,
|
||||
client: osClient,
|
||||
signer: signer,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// GetClient returns the underlying OpenSearch client for use by operation packages.
|
||||
func (c *Client) GetClient() *opensearch.Client {
|
||||
return c.client
|
||||
}
|
||||
|
||||
// validateConfig validates the client configuration.
|
||||
func validateConfig(cfg *Config) error {
|
||||
if cfg.Endpoint == "" {
|
||||
return errors.New("endpoint is required")
|
||||
}
|
||||
if cfg.Region == "" {
|
||||
return errors.New("region is required")
|
||||
}
|
||||
if cfg.AccessKey == "" {
|
||||
return errors.New("access key is required")
|
||||
}
|
||||
if cfg.SecretKey == "" {
|
||||
return errors.New("secret key is required")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
113
client/client_test.go
Normal file
113
client/client_test.go
Normal file
@@ -0,0 +1,113 @@
|
||||
package client
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestNewClient(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
config *Config
|
||||
wantErr error
|
||||
}{
|
||||
{
|
||||
name: "valid config",
|
||||
config: &Config{
|
||||
Endpoint: "https://example.com",
|
||||
Region: "us-east-1",
|
||||
AccessKey: "test-key",
|
||||
SecretKey: "test-secret",
|
||||
},
|
||||
wantErr: nil,
|
||||
},
|
||||
{
|
||||
name: "nil config",
|
||||
config: nil,
|
||||
wantErr: ErrInvalidConfig,
|
||||
},
|
||||
{
|
||||
name: "missing endpoint",
|
||||
config: &Config{
|
||||
Region: "us-east-1",
|
||||
AccessKey: "test-key",
|
||||
SecretKey: "test-secret",
|
||||
},
|
||||
wantErr: ErrInvalidConfig,
|
||||
},
|
||||
{
|
||||
name: "missing region",
|
||||
config: &Config{
|
||||
Endpoint: "https://example.com",
|
||||
AccessKey: "test-key",
|
||||
SecretKey: "test-secret",
|
||||
},
|
||||
wantErr: ErrInvalidConfig,
|
||||
},
|
||||
{
|
||||
name: "missing access key",
|
||||
config: &Config{
|
||||
Endpoint: "https://example.com",
|
||||
Region: "us-east-1",
|
||||
SecretKey: "test-secret",
|
||||
},
|
||||
wantErr: ErrInvalidConfig,
|
||||
},
|
||||
{
|
||||
name: "missing secret key",
|
||||
config: &Config{
|
||||
Endpoint: "https://example.com",
|
||||
Region: "us-east-1",
|
||||
AccessKey: "test-key",
|
||||
},
|
||||
wantErr: ErrInvalidConfig,
|
||||
},
|
||||
{
|
||||
name: "with custom timeout",
|
||||
config: &Config{
|
||||
Endpoint: "https://example.com",
|
||||
Region: "us-east-1",
|
||||
AccessKey: "test-key",
|
||||
SecretKey: "test-secret",
|
||||
Timeout: 60 * time.Second,
|
||||
},
|
||||
wantErr: nil,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got, err := NewClient(tt.config)
|
||||
if !errors.Is(err, tt.wantErr) {
|
||||
t.Errorf("NewClient() error = %v, wantErr %v", err, tt.wantErr)
|
||||
return
|
||||
}
|
||||
if tt.wantErr == nil && got == nil {
|
||||
t.Error("NewClient() returned nil client with no error")
|
||||
}
|
||||
if tt.wantErr != nil && got != nil {
|
||||
t.Error("NewClient() returned client with error")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestClient_GetClient(t *testing.T) {
|
||||
cfg := &Config{
|
||||
Endpoint: "https://example.com",
|
||||
Region: "us-east-1",
|
||||
AccessKey: "test-key",
|
||||
SecretKey: "test-secret",
|
||||
}
|
||||
|
||||
client, err := NewClient(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to create client: %v", err)
|
||||
}
|
||||
|
||||
osClient := client.GetClient()
|
||||
if osClient == nil {
|
||||
t.Error("GetClient() returned nil")
|
||||
}
|
||||
}
|
||||
47
config/config.go
Normal file
47
config/config.go
Normal file
@@ -0,0 +1,47 @@
|
||||
// Package config provides global configuration management for the application.
|
||||
// This is a simplified implementation suitable for the experimental phase.
|
||||
package config
|
||||
|
||||
import (
|
||||
"os"
|
||||
|
||||
"github.com/joho/godotenv"
|
||||
)
|
||||
|
||||
var (
|
||||
// Endpoint is the OpenSearch cluster endpoint URL.
|
||||
Endpoint string
|
||||
|
||||
// Region is the AWS region where the OpenSearch cluster is located.
|
||||
Region string
|
||||
|
||||
// AccessKey is the AWS access key ID for authentication.
|
||||
AccessKey string
|
||||
|
||||
// SecretKey is the AWS secret access key for authentication.
|
||||
SecretKey string
|
||||
)
|
||||
|
||||
// Load loads configuration from a .env file and then from environment variables.
|
||||
// If the file does not exist, it only loads from environment variables without error.
|
||||
func Load(filePath string) error {
|
||||
// Load .env file if it exists
|
||||
// This will set environment variables from the file
|
||||
_ = godotenv.Load(filePath)
|
||||
|
||||
// Initialize global config from environment variables
|
||||
Endpoint = os.Getenv("ES_ENDPOINT")
|
||||
AccessKey = os.Getenv("AWS_ACCESS_KEY_ID")
|
||||
SecretKey = os.Getenv("AWS_SECRET_ACCESS_KEY")
|
||||
Region = os.Getenv("AWS_REGION")
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// Init initializes the global configuration from environment variables.
|
||||
func Init() {
|
||||
Endpoint = os.Getenv("ES_ENDPOINT")
|
||||
AccessKey = os.Getenv("AWS_ACCESS_KEY_ID")
|
||||
SecretKey = os.Getenv("AWS_SECRET_ACCESS_KEY")
|
||||
Region = os.Getenv("AWS_REGION")
|
||||
}
|
||||
114
config/config_test.go
Normal file
114
config/config_test.go
Normal file
@@ -0,0 +1,114 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestInit(t *testing.T) {
|
||||
// Save original env vars
|
||||
origEndpoint := os.Getenv("ES_ENDPOINT")
|
||||
origRegion := os.Getenv("AWS_REGION")
|
||||
origAccessKey := os.Getenv("AWS_ACCESS_KEY_ID")
|
||||
origSecretKey := os.Getenv("AWS_SECRET_ACCESS_KEY")
|
||||
|
||||
// Restore original env vars after test
|
||||
defer func() {
|
||||
os.Setenv("ES_ENDPOINT", origEndpoint)
|
||||
os.Setenv("AWS_REGION", origRegion)
|
||||
os.Setenv("AWS_ACCESS_KEY_ID", origAccessKey)
|
||||
os.Setenv("AWS_SECRET_ACCESS_KEY", origSecretKey)
|
||||
}()
|
||||
|
||||
// Set test env vars
|
||||
os.Setenv("ES_ENDPOINT", "https://test.example.com")
|
||||
os.Setenv("AWS_REGION", "us-west-2")
|
||||
os.Setenv("AWS_ACCESS_KEY_ID", "test-access-key")
|
||||
os.Setenv("AWS_SECRET_ACCESS_KEY", "test-secret-key")
|
||||
|
||||
// Test Init
|
||||
Init()
|
||||
|
||||
if Endpoint != "https://test.example.com" {
|
||||
t.Errorf("Endpoint = %v, want %v", Endpoint, "https://test.example.com")
|
||||
}
|
||||
if Region != "us-west-2" {
|
||||
t.Errorf("Region = %v, want %v", Region, "us-west-2")
|
||||
}
|
||||
if AccessKey != "test-access-key" {
|
||||
t.Errorf("AccessKey = %v, want %v", AccessKey, "test-access-key")
|
||||
}
|
||||
if SecretKey != "test-secret-key" {
|
||||
t.Errorf("SecretKey = %v, want %v", SecretKey, "test-secret-key")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoad(t *testing.T) {
|
||||
// Save original env vars
|
||||
origEndpoint := os.Getenv("ES_ENDPOINT")
|
||||
origRegion := os.Getenv("AWS_REGION")
|
||||
origAccessKey := os.Getenv("AWS_ACCESS_KEY_ID")
|
||||
origSecretKey := os.Getenv("AWS_SECRET_ACCESS_KEY")
|
||||
|
||||
// Clear environment variables before test
|
||||
os.Unsetenv("ES_ENDPOINT")
|
||||
os.Unsetenv("AWS_REGION")
|
||||
os.Unsetenv("AWS_ACCESS_KEY_ID")
|
||||
os.Unsetenv("AWS_SECRET_ACCESS_KEY")
|
||||
|
||||
// Restore original env vars after test
|
||||
defer func() {
|
||||
if origEndpoint != "" {
|
||||
os.Setenv("ES_ENDPOINT", origEndpoint)
|
||||
}
|
||||
if origRegion != "" {
|
||||
os.Setenv("AWS_REGION", origRegion)
|
||||
}
|
||||
if origAccessKey != "" {
|
||||
os.Setenv("AWS_ACCESS_KEY_ID", origAccessKey)
|
||||
}
|
||||
if origSecretKey != "" {
|
||||
os.Setenv("AWS_SECRET_ACCESS_KEY", origSecretKey)
|
||||
}
|
||||
// Reload original config
|
||||
Init()
|
||||
}()
|
||||
|
||||
// Create a temporary .env file
|
||||
tmpFile := ".env.test"
|
||||
content := `ES_ENDPOINT=https://test-load.example.com
|
||||
AWS_REGION=eu-west-1
|
||||
AWS_ACCESS_KEY_ID=load-access-key
|
||||
AWS_SECRET_ACCESS_KEY=load-secret-key
|
||||
`
|
||||
if err := os.WriteFile(tmpFile, []byte(content), 0644); err != nil {
|
||||
t.Fatalf("failed to create test .env file: %v", err)
|
||||
}
|
||||
defer os.Remove(tmpFile)
|
||||
|
||||
// Test Load
|
||||
if err := Load(tmpFile); err != nil {
|
||||
t.Fatalf("Load() error = %v", err)
|
||||
}
|
||||
|
||||
if Endpoint != "https://test-load.example.com" {
|
||||
t.Errorf("Endpoint = %v, want %v", Endpoint, "https://test-load.example.com")
|
||||
}
|
||||
if Region != "eu-west-1" {
|
||||
t.Errorf("Region = %v, want %v", Region, "eu-west-1")
|
||||
}
|
||||
if AccessKey != "load-access-key" {
|
||||
t.Errorf("AccessKey = %v, want %v", AccessKey, "load-access-key")
|
||||
}
|
||||
if SecretKey != "load-secret-key" {
|
||||
t.Errorf("SecretKey = %v, want %v", SecretKey, "load-secret-key")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoad_FileNotFound(t *testing.T) {
|
||||
// Test with non-existent file - should not return error
|
||||
err := Load("non-existent-file.env")
|
||||
if err != nil {
|
||||
t.Errorf("Load() with non-existent file should not error, got %v", err)
|
||||
}
|
||||
}
|
||||
14
go.mod
Normal file
14
go.mod
Normal file
@@ -0,0 +1,14 @@
|
||||
module es-demo
|
||||
|
||||
go 1.25.3
|
||||
|
||||
require (
|
||||
github.com/aws/aws-sdk-go-v2 v1.39.6
|
||||
github.com/joho/godotenv v1.5.1
|
||||
github.com/opensearch-project/opensearch-go/v2 v2.3.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/aws/aws-sdk-go-v2/config v1.31.20 // indirect
|
||||
github.com/aws/smithy-go v1.23.2 // indirect
|
||||
)
|
||||
95
go.sum
Normal file
95
go.sum
Normal file
@@ -0,0 +1,95 @@
|
||||
github.com/aws/aws-sdk-go v1.44.263/go.mod h1:aVsgQcEevwlmQ7qHE9I3h+dtQgpqhFB+i8Phjh7fkwI=
|
||||
github.com/aws/aws-sdk-go-v2 v1.18.0/go.mod h1:uzbQtefpm44goOPmdKyAlXSNcwlRgF3ePWVW6EtJvvw=
|
||||
github.com/aws/aws-sdk-go-v2 v1.39.6 h1:2JrPCVgWJm7bm83BDwY5z8ietmeJUbh3O2ACnn+Xsqk=
|
||||
github.com/aws/aws-sdk-go-v2 v1.39.6/go.mod h1:c9pm7VwuW0UPxAEYGyTmyurVcNrbF6Rt/wixFqDhcjE=
|
||||
github.com/aws/aws-sdk-go-v2/config v1.18.25/go.mod h1:dZnYpD5wTW/dQF0rRNLVypB396zWCcPiBIvdvSWHEg4=
|
||||
github.com/aws/aws-sdk-go-v2/config v1.31.20 h1:/jWF4Wu90EhKCgjTdy1DGxcbcbNrjfBHvksEL79tfQc=
|
||||
github.com/aws/aws-sdk-go-v2/config v1.31.20/go.mod h1:95Hh1Tc5VYKL9NJ7tAkDcqeKt+MCXQB1hQZaRdJIZE0=
|
||||
github.com/aws/aws-sdk-go-v2/credentials v1.13.24/go.mod h1:jYPYi99wUOPIFi0rhiOvXeSEReVOzBqFNOX5bXYoG2o=
|
||||
github.com/aws/aws-sdk-go-v2/credentials v1.18.24 h1:iJ2FmPT35EaIB0+kMa6TnQ+PwG5A1prEdAw+PsMzfHg=
|
||||
github.com/aws/aws-sdk-go-v2/credentials v1.18.24/go.mod h1:U91+DrfjAiXPDEGYhh/x29o4p0qHX5HDqG7y5VViv64=
|
||||
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.3/go.mod h1:4Q0UFP0YJf0NrsEuEYHpM9fTSEVnD16Z3uyEF7J9JGM=
|
||||
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 h1:T1brd5dR3/fzNFAQch/iBKeX07/ffu/cLu+q+RuzEWk=
|
||||
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13/go.mod h1:Peg/GBAQ6JDt+RoBf4meB1wylmAipb7Kg2ZFakZTlwk=
|
||||
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.33/go.mod h1:7i0PF1ME/2eUPFcjkVIwq+DOygHEoK92t5cDqNgYbIw=
|
||||
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 h1:a+8/MLcWlIxo1lF9xaGt3J/u3yOZx+CdSveSNwjhD40=
|
||||
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13/go.mod h1:oGnKwIYZ4XttyU2JWxFrwvhF6YKiK/9/wmE3v3Iu9K8=
|
||||
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.27/go.mod h1:UrHnn3QV/d0pBZ6QBAEQcqFLf8FAzLmoUfPVIueOvoM=
|
||||
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 h1:HBSI2kDkMdWz4ZM7FjwE7e/pWDEZ+nR95x8Ztet1ooY=
|
||||
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13/go.mod h1:YE94ZoDArI7awZqJzBAZ3PDD2zSfuP7w6P2knOzIn8M=
|
||||
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.34/go.mod h1:Etz2dj6UHYuw+Xw830KfzCfWGMzqvUTCjUj5b76GVDc=
|
||||
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 h1:WKuaxf++XKWlHWu9ECbMlha8WOEGm0OUEZqm4K/Gcfk=
|
||||
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4/go.mod h1:ZWy7j6v1vWGmPReu0iSGvRiise4YI5SkR3OHKTZ6Wuc=
|
||||
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 h1:x2Ibm/Af8Fi+BH+Hsn9TXGdT+hKbDd5XOTZxTMxDk7o=
|
||||
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3/go.mod h1:IW1jwyrQgMdhisceG8fQLmQIydcT/jWY21rFhzgaKwo=
|
||||
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.27/go.mod h1:EOwBD4J4S5qYszS5/3DpkejfuK+Z5/1uzICfPaZLtqw=
|
||||
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 h1:kDqdFvMY4AtKoACfzIGD8A0+hbT41KTKF//gq7jITfM=
|
||||
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13/go.mod h1:lmKuogqSU3HzQCwZ9ZtcqOc5XGMqtDK7OIc2+DxiUEg=
|
||||
github.com/aws/aws-sdk-go-v2/service/sso v1.12.10/go.mod h1:ouy2P4z6sJN70fR3ka3wD3Ro3KezSxU6eKGQI2+2fjI=
|
||||
github.com/aws/aws-sdk-go-v2/service/sso v1.30.3 h1:NjShtS1t8r5LUfFVtFeI8xLAHQNTa7UI0VawXlrBMFQ=
|
||||
github.com/aws/aws-sdk-go-v2/service/sso v1.30.3/go.mod h1:fKvyjJcz63iL/ftA6RaM8sRCtN4r4zl4tjL3qw5ec7k=
|
||||
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.10/go.mod h1:AFvkxc8xfBe8XA+5St5XIHHrQQtkxqrRincx4hmMHOk=
|
||||
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.7 h1:gTsnx0xXNQ6SBbymoDvcoRHL+q4l/dAFsQuKfDWSaGc=
|
||||
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.7/go.mod h1:klO+ejMvYsB4QATfEOIXk8WAEwN4N0aBfJpvC+5SZBo=
|
||||
github.com/aws/aws-sdk-go-v2/service/sts v1.19.0/go.mod h1:BgQOMsg8av8jset59jelyPW7NoZcZXLVpDsXunGDrk8=
|
||||
github.com/aws/aws-sdk-go-v2/service/sts v1.40.2 h1:HK5ON3KmQV2HcAunnx4sKLB9aPf3gKGwVAf7xnx0QT0=
|
||||
github.com/aws/aws-sdk-go-v2/service/sts v1.40.2/go.mod h1:E19xDjpzPZC7LS2knI9E6BaRFDK43Eul7vd6rSq2HWk=
|
||||
github.com/aws/smithy-go v1.13.5/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
|
||||
github.com/aws/smithy-go v1.23.2 h1:Crv0eatJUQhaManss33hS5r40CG3ZFH+21XSkqMrIUM=
|
||||
github.com/aws/smithy-go v1.23.2/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
||||
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
|
||||
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
|
||||
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
|
||||
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
|
||||
github.com/opensearch-project/opensearch-go/v2 v2.3.0 h1:nQIEMr+A92CkhHrZgUhcfsrZjibvB3APXf2a1VwCmMQ=
|
||||
github.com/opensearch-project/opensearch-go/v2 v2.3.0/go.mod h1:8LDr9FCgUTVoT+5ESjc2+iaZuldqE+23Iq0r1XeNue8=
|
||||
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
||||
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
|
||||
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
|
||||
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
|
||||
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
|
||||
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
|
||||
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
|
||||
golang.org/x/net v0.1.0/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco=
|
||||
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
|
||||
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
|
||||
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
||||
golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
||||
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
||||
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
|
||||
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
|
||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
109
main.go
Normal file
109
main.go
Normal file
@@ -0,0 +1,109 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"es-demo/client"
|
||||
"es-demo/config"
|
||||
"es-demo/operations/cluster"
|
||||
"es-demo/operations/index"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// Load configuration from .env file and environment variables
|
||||
if err := config.Load(".env"); err != nil {
|
||||
log.Fatalf("Failed to load configuration: %v", err)
|
||||
}
|
||||
|
||||
// Create client configuration
|
||||
cfg := &client.Config{
|
||||
Endpoint: config.Endpoint,
|
||||
Region: config.Region,
|
||||
AccessKey: config.AccessKey,
|
||||
SecretKey: config.SecretKey,
|
||||
Timeout: 30 * time.Second,
|
||||
}
|
||||
|
||||
// Create OpenSearch client
|
||||
osClient, err := client.NewClient(cfg)
|
||||
if err != nil {
|
||||
log.Fatalf("Failed to create OpenSearch client: %v", err)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
|
||||
// Get cluster information
|
||||
info, err := cluster.GetInfo(ctx, osClient)
|
||||
if err != nil {
|
||||
log.Fatalf("Failed to get cluster info: %v", err)
|
||||
}
|
||||
|
||||
// Display cluster information
|
||||
fmt.Printf("=== Cluster Information ===\n")
|
||||
fmt.Printf("Cluster Name: %s\n", info.ClusterName)
|
||||
fmt.Printf("Cluster UUID: %s\n", info.ClusterUUID)
|
||||
fmt.Printf("Version: %s\n", info.Version.Number)
|
||||
fmt.Printf("Lucene Version: %s\n", info.Version.LuceneVersion)
|
||||
fmt.Printf("Tagline: %s\n\n", info.Tagline)
|
||||
|
||||
// Example: Create an index template
|
||||
template := &index.Template{
|
||||
IndexPatterns: []string{"logs-*"},
|
||||
Settings: map[string]any{
|
||||
"number_of_shards": 1,
|
||||
"number_of_replicas": 1,
|
||||
},
|
||||
Mappings: map[string]any{
|
||||
"properties": map[string]any{
|
||||
"timestamp": map[string]any{
|
||||
"type": "date",
|
||||
},
|
||||
"message": map[string]any{
|
||||
"type": "text",
|
||||
},
|
||||
},
|
||||
},
|
||||
Priority: 100,
|
||||
}
|
||||
|
||||
fmt.Printf("=== Index Template Operations ===\n")
|
||||
|
||||
// Put template
|
||||
if err := index.PutTemplate(ctx, osClient, "logs-template", template); err != nil {
|
||||
log.Printf("Warning: Failed to create template: %v\n", err)
|
||||
} else {
|
||||
fmt.Printf("✓ Created template: logs-template\n")
|
||||
}
|
||||
|
||||
// Get template
|
||||
retrievedTemplate, err := index.GetTemplate(ctx, osClient, "logs-template")
|
||||
if err != nil {
|
||||
log.Printf("Warning: Failed to get template: %v\n", err)
|
||||
} else {
|
||||
fmt.Printf("✓ Retrieved template: %+v\n", retrievedTemplate.IndexPatterns)
|
||||
}
|
||||
|
||||
// List templates
|
||||
templates, err := index.ListTemplates(ctx, osClient)
|
||||
if err != nil {
|
||||
log.Printf("Warning: Failed to list templates: %v\n", err)
|
||||
} else {
|
||||
fmt.Printf("✓ Total templates: %d\n", len(templates))
|
||||
for name := range templates {
|
||||
fmt.Printf(" - %s\n", name)
|
||||
}
|
||||
}
|
||||
|
||||
// Delete template (commented out to avoid cleanup in demo)
|
||||
// if err := index.DeleteTemplate(ctx, osClient, "logs-template"); err != nil {
|
||||
// log.Printf("Warning: Failed to delete template: %v\n", err)
|
||||
// } else {
|
||||
// fmt.Printf("✓ Deleted template: logs-template\n")
|
||||
// }
|
||||
|
||||
fmt.Printf("\nDemo completed successfully!\n")
|
||||
}
|
||||
58
operations/cluster/cluster.go
Normal file
58
operations/cluster/cluster.go
Normal file
@@ -0,0 +1,58 @@
|
||||
// Package cluster provides cluster-level operations for OpenSearch administration.
|
||||
package cluster
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"es-demo/client"
|
||||
|
||||
"github.com/opensearch-project/opensearch-go/v2/opensearchapi"
|
||||
)
|
||||
|
||||
// Info represents essential information about the OpenSearch cluster.
|
||||
type Info struct {
|
||||
Name string `json:"name"`
|
||||
ClusterName string `json:"cluster_name"`
|
||||
ClusterUUID string `json:"cluster_uuid"`
|
||||
Version struct {
|
||||
Number string `json:"number"`
|
||||
BuildType string `json:"build_type"`
|
||||
BuildHash string `json:"build_hash"`
|
||||
BuildDate string `json:"build_date"`
|
||||
BuildSnapshot bool `json:"build_snapshot"`
|
||||
LuceneVersion string `json:"lucene_version"`
|
||||
MinimumWireCompatibilityVersion string `json:"minimum_wire_compatibility_version"`
|
||||
MinimumIndexCompatibilityVersion string `json:"minimum_index_compatibility_version"`
|
||||
} `json:"version"`
|
||||
Tagline string `json:"tagline"`
|
||||
}
|
||||
|
||||
// GetInfo retrieves essential information about the OpenSearch cluster.
|
||||
// It returns the cluster information or an error if the request fails.
|
||||
func GetInfo(ctx context.Context, c *client.Client) (*Info, error) {
|
||||
req := opensearchapi.InfoRequest{}
|
||||
|
||||
res, err := req.Do(ctx, c.GetClient())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to execute info request: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if closeErr := res.Body.Close(); closeErr != nil {
|
||||
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
|
||||
}
|
||||
}()
|
||||
|
||||
if res.IsError() {
|
||||
return nil, fmt.Errorf("request failed with status: %s", res.Status())
|
||||
}
|
||||
|
||||
var info Info
|
||||
if err := json.NewDecoder(res.Body).Decode(&info); err != nil {
|
||||
return nil, fmt.Errorf("failed to decode cluster info: %w", err)
|
||||
}
|
||||
|
||||
return &info, nil
|
||||
}
|
||||
40
operations/cluster/cluster_test.go
Normal file
40
operations/cluster/cluster_test.go
Normal file
@@ -0,0 +1,40 @@
|
||||
package cluster
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"es-demo/client"
|
||||
)
|
||||
|
||||
func TestGetInfo(t *testing.T) {
|
||||
// This is a unit test that doesn't require real connection
|
||||
// Integration tests should be run separately
|
||||
if testing.Short() {
|
||||
t.Skip("skipping integration test in short mode")
|
||||
}
|
||||
|
||||
cfg := &client.Config{
|
||||
Endpoint: "https://example.com",
|
||||
Region: "us-east-1",
|
||||
AccessKey: "test-key",
|
||||
SecretKey: "test-secret",
|
||||
}
|
||||
|
||||
c, err := client.NewClient(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to create client: %v", err)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
|
||||
// This will fail without a real cluster, which is expected in unit tests
|
||||
_, err = GetInfo(ctx, c)
|
||||
// We just verify the method signature is correct
|
||||
// Real integration tests would check actual responses
|
||||
if err == nil {
|
||||
t.Log("GetInfo succeeded (unexpected in unit test)")
|
||||
}
|
||||
}
|
||||
217
operations/index/template.go
Normal file
217
operations/index/template.go
Normal file
@@ -0,0 +1,217 @@
|
||||
// Package index provides index-level operations for OpenSearch.
|
||||
package index
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
|
||||
"es-demo/client"
|
||||
|
||||
"github.com/opensearch-project/opensearch-go/v2/opensearchapi"
|
||||
)
|
||||
|
||||
var (
|
||||
// ErrInvalidTemplate is returned when the template configuration is invalid.
|
||||
ErrInvalidTemplate = errors.New("invalid template configuration")
|
||||
|
||||
// ErrTemplateNotFound is returned when the specified template does not exist.
|
||||
ErrTemplateNotFound = errors.New("template not found")
|
||||
)
|
||||
|
||||
// Template represents an OpenSearch index template.
|
||||
type Template struct {
|
||||
// IndexPatterns defines the index patterns this template applies to.
|
||||
IndexPatterns []string `json:"index_patterns"`
|
||||
|
||||
// Settings contains index settings like number of shards and replicas.
|
||||
Settings map[string]any `json:"settings,omitempty"`
|
||||
|
||||
// Mappings defines the field mappings for the index.
|
||||
Mappings map[string]any `json:"mappings,omitempty"`
|
||||
|
||||
// Aliases defines index aliases.
|
||||
Aliases map[string]any `json:"aliases,omitempty"`
|
||||
|
||||
// Priority determines template precedence when multiple templates match.
|
||||
Priority int `json:"priority,omitempty"`
|
||||
|
||||
// Version is used for external version management.
|
||||
Version int `json:"version,omitempty"`
|
||||
}
|
||||
|
||||
// TemplateResponse represents the response when getting a template.
|
||||
type TemplateResponse struct {
|
||||
IndexTemplates []struct {
|
||||
Name string `json:"name"`
|
||||
IndexTemplate Template `json:"index_template"`
|
||||
} `json:"index_templates"`
|
||||
}
|
||||
|
||||
// PutTemplate creates or updates an index template.
|
||||
func PutTemplate(ctx context.Context, c *client.Client, name string, template *Template) error {
|
||||
if err := validateTemplate(template); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if name == "" {
|
||||
return fmt.Errorf("%w: template name is required", ErrInvalidTemplate)
|
||||
}
|
||||
|
||||
body, err := json.Marshal(template)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to marshal template: %w", err)
|
||||
}
|
||||
|
||||
req := opensearchapi.IndicesPutIndexTemplateRequest{
|
||||
Name: name,
|
||||
Body: bytes.NewReader(body),
|
||||
}
|
||||
|
||||
res, err := req.Do(ctx, c.GetClient())
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to execute put template request: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if closeErr := res.Body.Close(); closeErr != nil {
|
||||
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
|
||||
}
|
||||
}()
|
||||
|
||||
if res.IsError() {
|
||||
bodyBytes, _ := io.ReadAll(res.Body)
|
||||
return fmt.Errorf("put template failed with status %s: %s", res.Status(), string(bodyBytes))
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetTemplate retrieves an index template by name.
|
||||
func GetTemplate(ctx context.Context, c *client.Client, name string) (*Template, error) {
|
||||
if name == "" {
|
||||
return nil, fmt.Errorf("%w: template name is required", ErrInvalidTemplate)
|
||||
}
|
||||
|
||||
req := opensearchapi.IndicesGetIndexTemplateRequest{
|
||||
Name: []string{name},
|
||||
}
|
||||
|
||||
res, err := req.Do(ctx, c.GetClient())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to execute get template request: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if closeErr := res.Body.Close(); closeErr != nil {
|
||||
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
|
||||
}
|
||||
}()
|
||||
|
||||
if res.IsError() {
|
||||
if res.StatusCode == 404 {
|
||||
return nil, ErrTemplateNotFound
|
||||
}
|
||||
bodyBytes, _ := io.ReadAll(res.Body)
|
||||
return nil, fmt.Errorf("get template failed with status %s: %s", res.Status(), string(bodyBytes))
|
||||
}
|
||||
|
||||
var response TemplateResponse
|
||||
if err := json.NewDecoder(res.Body).Decode(&response); err != nil {
|
||||
return nil, fmt.Errorf("failed to decode template response: %w", err)
|
||||
}
|
||||
|
||||
if len(response.IndexTemplates) == 0 {
|
||||
return nil, ErrTemplateNotFound
|
||||
}
|
||||
|
||||
return &response.IndexTemplates[0].IndexTemplate, nil
|
||||
}
|
||||
|
||||
// DeleteTemplate deletes an index template.
|
||||
func DeleteTemplate(ctx context.Context, c *client.Client, name string) error {
|
||||
if name == "" {
|
||||
return fmt.Errorf("%w: template name is required", ErrInvalidTemplate)
|
||||
}
|
||||
|
||||
req := opensearchapi.IndicesDeleteIndexTemplateRequest{
|
||||
Name: name,
|
||||
}
|
||||
|
||||
res, err := req.Do(ctx, c.GetClient())
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to execute delete template request: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if closeErr := res.Body.Close(); closeErr != nil {
|
||||
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
|
||||
}
|
||||
}()
|
||||
|
||||
if res.IsError() {
|
||||
if res.StatusCode == 404 {
|
||||
return ErrTemplateNotFound
|
||||
}
|
||||
bodyBytes, _ := io.ReadAll(res.Body)
|
||||
return fmt.Errorf("delete template failed with status %s: %s", res.Status(), string(bodyBytes))
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// ListTemplates retrieves all index templates or templates matching a pattern.
|
||||
func ListTemplates(ctx context.Context, c *client.Client, names ...string) (map[string]*Template, error) {
|
||||
var nameList []string
|
||||
if len(names) > 0 {
|
||||
nameList = names
|
||||
}
|
||||
|
||||
req := opensearchapi.IndicesGetIndexTemplateRequest{
|
||||
Name: nameList,
|
||||
}
|
||||
|
||||
res, err := req.Do(ctx, c.GetClient())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to execute list templates request: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if closeErr := res.Body.Close(); closeErr != nil {
|
||||
fmt.Fprintf(os.Stderr, "warning: failed to close response body: %v\n", closeErr)
|
||||
}
|
||||
}()
|
||||
|
||||
if res.IsError() {
|
||||
if res.StatusCode == 404 {
|
||||
return make(map[string]*Template), nil
|
||||
}
|
||||
bodyBytes, _ := io.ReadAll(res.Body)
|
||||
return nil, fmt.Errorf("list templates failed with status %s: %s", res.Status(), string(bodyBytes))
|
||||
}
|
||||
|
||||
var response TemplateResponse
|
||||
if err := json.NewDecoder(res.Body).Decode(&response); err != nil {
|
||||
return nil, fmt.Errorf("failed to decode templates response: %w", err)
|
||||
}
|
||||
|
||||
templates := make(map[string]*Template)
|
||||
for _, item := range response.IndexTemplates {
|
||||
templates[item.Name] = &item.IndexTemplate
|
||||
}
|
||||
|
||||
return templates, nil
|
||||
}
|
||||
|
||||
// validateTemplate validates the template configuration.
|
||||
func validateTemplate(template *Template) error {
|
||||
if template == nil {
|
||||
return fmt.Errorf("%w: template cannot be nil", ErrInvalidTemplate)
|
||||
}
|
||||
|
||||
if len(template.IndexPatterns) == 0 {
|
||||
return fmt.Errorf("%w: index patterns are required", ErrInvalidTemplate)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
180
operations/index/template_test.go
Normal file
180
operations/index/template_test.go
Normal file
@@ -0,0 +1,180 @@
|
||||
package index
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"es-demo/client"
|
||||
)
|
||||
|
||||
func TestPutTemplate(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("skipping integration test in short mode")
|
||||
}
|
||||
|
||||
cfg := &client.Config{
|
||||
Endpoint: "https://example.com",
|
||||
Region: "us-east-1",
|
||||
AccessKey: "test-key",
|
||||
SecretKey: "test-secret",
|
||||
}
|
||||
|
||||
c, err := client.NewClient(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to create client: %v", err)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
|
||||
template := &Template{
|
||||
IndexPatterns: []string{"test-*"},
|
||||
Settings: map[string]any{
|
||||
"number_of_shards": 1,
|
||||
"number_of_replicas": 0,
|
||||
},
|
||||
Mappings: map[string]any{
|
||||
"properties": map[string]any{
|
||||
"timestamp": map[string]any{
|
||||
"type": "date",
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
// This will fail without a real cluster
|
||||
err = PutTemplate(ctx, c, "test-template", template)
|
||||
if err == nil {
|
||||
t.Log("PutTemplate succeeded (unexpected in unit test)")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGetTemplate(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("skipping integration test in short mode")
|
||||
}
|
||||
|
||||
cfg := &client.Config{
|
||||
Endpoint: "https://example.com",
|
||||
Region: "us-east-1",
|
||||
AccessKey: "test-key",
|
||||
SecretKey: "test-secret",
|
||||
}
|
||||
|
||||
c, err := client.NewClient(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to create client: %v", err)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
|
||||
// This will fail without a real cluster
|
||||
_, err = GetTemplate(ctx, c, "test-template")
|
||||
if err == nil {
|
||||
t.Log("GetTemplate succeeded (unexpected in unit test)")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteTemplate(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("skipping integration test in short mode")
|
||||
}
|
||||
|
||||
cfg := &client.Config{
|
||||
Endpoint: "https://example.com",
|
||||
Region: "us-east-1",
|
||||
AccessKey: "test-key",
|
||||
SecretKey: "test-secret",
|
||||
}
|
||||
|
||||
c, err := client.NewClient(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to create client: %v", err)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
|
||||
// This will fail without a real cluster
|
||||
err = DeleteTemplate(ctx, c, "test-template")
|
||||
if err == nil {
|
||||
t.Log("DeleteTemplate succeeded (unexpected in unit test)")
|
||||
}
|
||||
}
|
||||
|
||||
func TestListTemplates(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("skipping integration test in short mode")
|
||||
}
|
||||
|
||||
cfg := &client.Config{
|
||||
Endpoint: "https://example.com",
|
||||
Region: "us-east-1",
|
||||
AccessKey: "test-key",
|
||||
SecretKey: "test-secret",
|
||||
}
|
||||
|
||||
c, err := client.NewClient(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to create client: %v", err)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
|
||||
// This will fail without a real cluster
|
||||
_, err = ListTemplates(ctx, c)
|
||||
if err == nil {
|
||||
t.Log("ListTemplates succeeded (unexpected in unit test)")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTemplate_Validate(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
template *Template
|
||||
wantErr error
|
||||
}{
|
||||
{
|
||||
name: "valid template",
|
||||
template: &Template{
|
||||
IndexPatterns: []string{"logs-*"},
|
||||
Settings: map[string]any{
|
||||
"number_of_shards": 1,
|
||||
},
|
||||
},
|
||||
wantErr: nil,
|
||||
},
|
||||
{
|
||||
name: "nil template",
|
||||
template: nil,
|
||||
wantErr: ErrInvalidTemplate,
|
||||
},
|
||||
{
|
||||
name: "empty index patterns",
|
||||
template: &Template{
|
||||
IndexPatterns: []string{},
|
||||
},
|
||||
wantErr: ErrInvalidTemplate,
|
||||
},
|
||||
{
|
||||
name: "nil index patterns",
|
||||
template: &Template{
|
||||
IndexPatterns: nil,
|
||||
},
|
||||
wantErr: ErrInvalidTemplate,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
err := validateTemplate(tt.template)
|
||||
if !errors.Is(err, tt.wantErr) {
|
||||
t.Errorf("validateTemplate() error = %v, wantErr %v", err, tt.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
213
test.ps1
Normal file
213
test.ps1
Normal file
@@ -0,0 +1,213 @@
|
||||
# 测试流水线脚本
|
||||
# 用途: 执行完整的代码质量检查流程
|
||||
# 包括: Linting -> Build -> Test -> Coverage -> Cleanup
|
||||
|
||||
param(
|
||||
[switch]$SkipLint,
|
||||
[switch]$SkipBuild,
|
||||
[switch]$SkipTest,
|
||||
[switch]$Verbose
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$Script:FailureCount = 0
|
||||
|
||||
# 颜色输出函数
|
||||
function Write-Success { param($Message) Write-Host "✓ $Message" -ForegroundColor Green }
|
||||
function Write-Failure { param($Message) Write-Host "✗ $Message" -ForegroundColor Red; $Script:FailureCount++ }
|
||||
function Write-Info { param($Message) Write-Host "→ $Message" -ForegroundColor Cyan }
|
||||
function Write-Section { param($Message) Write-Host "`n=== $Message ===" -ForegroundColor Yellow }
|
||||
|
||||
# 清理函数
|
||||
function Cleanup {
|
||||
Write-Section "Cleanup"
|
||||
|
||||
# 删除构建产物
|
||||
$artifacts = @("*.exe", "*.test", "coverage.out", "coverage.html")
|
||||
foreach ($pattern in $artifacts) {
|
||||
Get-ChildItem -Path . -Filter $pattern -Recurse -ErrorAction SilentlyContinue |
|
||||
Remove-Item -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
# 清理测试生成的临时文件
|
||||
Get-ChildItem -Path . -Filter ".env.test" -Recurse -ErrorAction SilentlyContinue |
|
||||
Remove-Item -Force -ErrorAction SilentlyContinue
|
||||
|
||||
Write-Success "Cleanup completed"
|
||||
}
|
||||
|
||||
# Linting
|
||||
function Run-Linting {
|
||||
Write-Section "Code Linting"
|
||||
|
||||
try {
|
||||
Write-Info "Running golangci-lint..."
|
||||
|
||||
# 检查 golangci-lint 是否安装
|
||||
$lintPath = Get-Command golangci-lint -ErrorAction SilentlyContinue
|
||||
if (-not $lintPath) {
|
||||
Write-Failure "golangci-lint not found. Please install it first."
|
||||
return $false
|
||||
}
|
||||
|
||||
# 运行 linter
|
||||
$output = & golangci-lint run --config .golangci.yml ./... 2>&1
|
||||
|
||||
if ($LASTEXITCODE -eq 0) {
|
||||
Write-Success "Linting passed"
|
||||
return $true
|
||||
} else {
|
||||
Write-Failure "Linting failed"
|
||||
if ($Verbose) {
|
||||
Write-Host $output -ForegroundColor Red
|
||||
}
|
||||
return $false
|
||||
}
|
||||
}
|
||||
catch {
|
||||
Write-Failure "Linting error: $_"
|
||||
return $false
|
||||
}
|
||||
}
|
||||
|
||||
# Build
|
||||
function Run-Build {
|
||||
Write-Section "Build"
|
||||
|
||||
try {
|
||||
Write-Info "Building project..."
|
||||
|
||||
# 使用 NUL 设备作为输出(Windows 的 /dev/null)
|
||||
$env:GOOS = ""
|
||||
$env:GOARCH = ""
|
||||
|
||||
# 构建但不保存二进制文件
|
||||
$output = go build -o NUL . 2>&1
|
||||
|
||||
if ($LASTEXITCODE -eq 0) {
|
||||
Write-Success "Build successful"
|
||||
return $true
|
||||
} else {
|
||||
Write-Failure "Build failed"
|
||||
if ($Verbose) {
|
||||
Write-Host $output -ForegroundColor Red
|
||||
}
|
||||
return $false
|
||||
}
|
||||
}
|
||||
catch {
|
||||
Write-Failure "Build error: $_"
|
||||
return $false
|
||||
}
|
||||
}
|
||||
|
||||
# Tests
|
||||
function Run-Tests {
|
||||
Write-Section "Tests"
|
||||
|
||||
try {
|
||||
Write-Info "Running unit tests..."
|
||||
|
||||
# 运行短测试(跳过集成测试)
|
||||
# 注意: Windows 下不启用 race detector (需要 cgo)
|
||||
# PowerShell 需要使用引号包裹带等号的参数
|
||||
$testOutput = go test -v -short "-coverprofile=coverage.out" ./... 2>&1
|
||||
|
||||
if ($LASTEXITCODE -eq 0) {
|
||||
Write-Success "All tests passed"
|
||||
|
||||
# 显示覆盖率摘要
|
||||
Write-Info "Coverage summary:"
|
||||
$coverageOutput = go tool cover -func coverage.out 2>&1
|
||||
$totalLine = $coverageOutput | Select-String "total:"
|
||||
if ($totalLine) {
|
||||
Write-Host " $totalLine" -ForegroundColor White
|
||||
}
|
||||
|
||||
# 可选:生成 HTML 覆盖率报告
|
||||
if ($Verbose) {
|
||||
go tool cover -html=coverage.out -o coverage.html
|
||||
Write-Success "Coverage report generated: coverage.html"
|
||||
}
|
||||
|
||||
return $true
|
||||
} else {
|
||||
Write-Failure "Tests failed"
|
||||
if ($Verbose) {
|
||||
Write-Host $output -ForegroundColor Red
|
||||
}
|
||||
return $false
|
||||
}
|
||||
}
|
||||
catch {
|
||||
Write-Failure "Test error: $_"
|
||||
return $false
|
||||
}
|
||||
}
|
||||
|
||||
# 主流程
|
||||
function Main {
|
||||
Write-Host @"
|
||||
╔════════════════════════════════════════╗
|
||||
║ ES-Demo Test Pipeline v1.0 ║
|
||||
╚════════════════════════════════════════╝
|
||||
"@ -ForegroundColor Magenta
|
||||
|
||||
$startTime = Get-Date
|
||||
|
||||
# 执行流水线步骤
|
||||
$results = @{
|
||||
Lint = $true
|
||||
Build = $true
|
||||
Test = $true
|
||||
}
|
||||
|
||||
if (-not $SkipLint) {
|
||||
$results.Lint = Run-Linting
|
||||
} else {
|
||||
Write-Info "Skipping linting"
|
||||
}
|
||||
|
||||
if (-not $SkipBuild) {
|
||||
$results.Build = Run-Build
|
||||
} else {
|
||||
Write-Info "Skipping build"
|
||||
}
|
||||
|
||||
if (-not $SkipTest) {
|
||||
$results.Test = Run-Tests
|
||||
} else {
|
||||
Write-Info "Skipping tests"
|
||||
}
|
||||
|
||||
# 清理
|
||||
Cleanup
|
||||
|
||||
# 结果汇总
|
||||
$endTime = Get-Date
|
||||
$duration = $endTime - $startTime
|
||||
|
||||
Write-Section "Summary"
|
||||
Write-Host "Lint: " -NoNewline
|
||||
if ($results.Lint) { Write-Success "PASS" } else { Write-Failure "FAIL" }
|
||||
|
||||
Write-Host "Build: " -NoNewline
|
||||
if ($results.Build) { Write-Success "PASS" } else { Write-Failure "FAIL" }
|
||||
|
||||
Write-Host "Test: " -NoNewline
|
||||
if ($results.Test) { Write-Success "PASS" } else { Write-Failure "FAIL" }
|
||||
|
||||
Write-Host "`nTotal time: $($duration.TotalSeconds) seconds" -ForegroundColor Cyan
|
||||
|
||||
# 返回退出码
|
||||
if ($Script:FailureCount -gt 0) {
|
||||
Write-Host "`n❌ Pipeline FAILED with $Script:FailureCount error(s)" -ForegroundColor Red
|
||||
exit 1
|
||||
} else {
|
||||
Write-Host "`n✅ Pipeline PASSED" -ForegroundColor Green
|
||||
exit 0
|
||||
}
|
||||
}
|
||||
|
||||
# 运行主流程
|
||||
Main
|
||||
Reference in New Issue
Block a user