提交
This commit is contained in:
1 parent
4acc17efa4
commit
48f15745be
6 files changed
+788
No files matched your search
+116
@@ -0,0 +1,116 @@
|
||||
# 学生学籍号刷新脚本
|
||||
|
||||
## 📋 脚本说明
|
||||
|
||||
本脚本用于自动刷新学生信息表(`xg_student_msg`)中的学籍号字段(`ext_field2`)。
|
||||
|
||||
**刷新规则**: `ext_field2 = 'G' + id_card`
|
||||
|
||||
## 🚀 快速开始
|
||||
|
||||
### 1. 一次性刷新(推荐)
|
||||
|
||||
```bash
|
||||
# 进入脚本目录
|
||||
cd 学籍号刷新
|
||||
|
||||
# 执行一次性刷新所有学籍号
|
||||
./run_refresh.sh once
|
||||
# 或者直接运行
|
||||
./process_student_id_card_refresh.sh once
|
||||
```
|
||||
|
||||
### 2. 持续监控模式
|
||||
|
||||
```bash
|
||||
# 前台执行(会一直运行)
|
||||
./run_refresh.sh monitor
|
||||
|
||||
# 后台执行(推荐方式)
|
||||
./run_refresh.sh bg
|
||||
```
|
||||
|
||||
## 📊 执行结果
|
||||
|
||||
最新执行结果(2025-12-26):
|
||||
- **总学生数**: 13,884
|
||||
- **已生成学籍号**: 13,883
|
||||
- **待刷新**: 1
|
||||
|
||||
### 📈 优化说明
|
||||
|
||||
**v1.0.2 更新 (2025-12-26)**:
|
||||
- ✅ **后台执行**: 新增`bg`模式,支持一键后台启动守护进程
|
||||
- ✅ **用户体验**: 提供PID显示、状态检查和停止命令提示
|
||||
|
||||
**v1.0.1 更新 (2025-12-26)**:
|
||||
- ✅ **简化逻辑**: 移除不必要的预查询,直接执行UPDATE语句
|
||||
- ✅ **提高效率**: 使用`SELECT ROW_COUNT()`获取受影响行数,一次查询完成
|
||||
- ✅ **减少数据库压力**: 减少不必要的统计查询
|
||||
|
||||
## 📁 文件结构
|
||||
|
||||
```
|
||||
学籍号刷新/
|
||||
├── data.conf # 开发环境配置
|
||||
├── data.conf.prod # 生产环境配置
|
||||
├── process_student_id_card_refresh.sh # 主脚本
|
||||
├── run_refresh.sh # 快速执行脚本
|
||||
├── README.md # 使用说明
|
||||
├── 学生学籍号刷新脚本设计说明.md # 详细设计文档
|
||||
└── student_id_card_refresh/ # 日志目录
|
||||
└── *.log # 执行日志
|
||||
```
|
||||
|
||||
## ⚙️ 配置说明
|
||||
|
||||
### data.conf 参数
|
||||
|
||||
```bash
|
||||
# 数据库配置
|
||||
DB_HOST="47.99.164.66"
|
||||
DB_PORT="3306"
|
||||
DB_NAME="hzdyjs_xg"
|
||||
DB_USER="root"
|
||||
DB_PASS="your_password"
|
||||
|
||||
# 处理配置
|
||||
BATCH_SIZE=500 # 批次大小
|
||||
CYCLE_INTERVAL=600 # 检查间隔(秒,默认10分钟)
|
||||
```
|
||||
|
||||
## 📈 验证结果
|
||||
|
||||
```sql
|
||||
-- 统计学籍号生成情况
|
||||
SELECT
|
||||
COUNT(*) as total_students,
|
||||
SUM(CASE WHEN ext_field2 LIKE 'G%' THEN 1 ELSE 0 END) as with_id_card,
|
||||
SUM(CASE WHEN ext_field2 IS NULL OR ext_field2 = '' THEN 1 ELSE 0 END) as need_refresh
|
||||
FROM xg_student_msg
|
||||
WHERE deleted = 0;
|
||||
|
||||
-- 查看学籍号样例
|
||||
SELECT student_no, name, id_card, ext_field2
|
||||
FROM xg_student_msg
|
||||
WHERE ext_field2 LIKE 'G%'
|
||||
AND deleted = 0
|
||||
LIMIT 5;
|
||||
```
|
||||
|
||||
## 🔒 安全注意
|
||||
|
||||
1. **备份数据**: 执行前务必备份数据库
|
||||
2. **测试环境**: 先在测试环境验证
|
||||
3. **权限控制**: 确保数据库用户有UPDATE权限
|
||||
|
||||
## 📞 技术支持
|
||||
|
||||
如有问题,请参考:
|
||||
- [学生学籍号刷新脚本设计说明.md](学生学籍号刷新脚本设计说明.md)
|
||||
- 项目技术团队
|
||||
|
||||
---
|
||||
|
||||
**最后更新**: 2025-12-26
|
||||
**脚本版本**: v1.0.2
|
||||
@@ -0,0 +1,17 @@
|
||||
# Database configuration
|
||||
DB_HOST="47.99.164.66"
|
||||
DB_PORT="3306"
|
||||
DB_NAME="hzdyjs_xg"
|
||||
DB_USER="root"
|
||||
DB_PASS="hn02le.34lkdLKD"
|
||||
|
||||
# Script configuration
|
||||
LOG_DIR="${SCRIPT_DIR}/student_id_card_refresh"
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
LOCK_FILE="${SCRIPT_DIR}/script.lock"
|
||||
|
||||
# Processing configuration
|
||||
BATCH_SIZE=500 # 学籍号刷新批次大小,较大一些提高效率
|
||||
MAX_RETRY=3
|
||||
RETRY_DELAY=5
|
||||
CYCLE_INTERVAL=600 # 循环间隔时间(秒),默认10分钟
|
||||
@@ -0,0 +1,17 @@
|
||||
# Database configuration - PRODUCTION
|
||||
DB_HOST="172.16.0.67"
|
||||
DB_PORT="3306"
|
||||
DB_NAME="hzdyjs_xg"
|
||||
DB_USER="root"
|
||||
DB_PASS="Hzdyjs@Root@123456"
|
||||
|
||||
# Script configuration
|
||||
LOG_DIR="${SCRIPT_DIR}/student_id_card_refresh"
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
LOCK_FILE="${SCRIPT_DIR}/script.lock"
|
||||
|
||||
# Processing configuration
|
||||
BATCH_SIZE=500 # 学籍号刷新批次大小,较大一些提高效率
|
||||
MAX_RETRY=3
|
||||
RETRY_DELAY=5
|
||||
CYCLE_INTERVAL=600 # 循环间隔时间(秒),默认10分钟
|
||||
Executable
+249
@@ -0,0 +1,249 @@
|
||||
#!/bin/bash
|
||||
|
||||
# ===========================================
|
||||
# 学生学籍号刷新脚本 v1.0.2
|
||||
# 处理类型: ID_CARD_REFRESH(学籍号刷新)
|
||||
# 刷新规则: ext_field2 = 'G' + id_card
|
||||
# 优化: 直接UPDATE + ROW_COUNT(),无需预查询
|
||||
# 新增: 支持run_refresh.sh bg模式后台执行
|
||||
# ===========================================
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Load configuration
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
CONFIG_FILE="${SCRIPT_DIR}/data.conf.prod"
|
||||
|
||||
if [[ ! -f "$CONFIG_FILE" ]]; then
|
||||
echo "错误:配置文件 $CONFIG_FILE 未找到!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
source "$CONFIG_FILE"
|
||||
|
||||
# Create log directory
|
||||
mkdir -p "$LOG_DIR"
|
||||
|
||||
# Log file
|
||||
LOG_FILE="${LOG_DIR}/process_student_id_card_refresh_$(date +%Y%m%d_%H%M%S).log"
|
||||
|
||||
# Lock file to prevent multiple instances
|
||||
LOCK_FILE="${SCRIPT_DIR}/process_student_id_card_refresh.lock"
|
||||
|
||||
# Functions
|
||||
log() {
|
||||
local level="$1"
|
||||
local message="$2"
|
||||
local timestamp=$(date '+%Y-%m-%d %H:%M:%S')
|
||||
echo "[$timestamp] [$level] $message" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
error_exit() {
|
||||
local message="$1"
|
||||
log "ERROR" "$message"
|
||||
# Remove lock file
|
||||
[[ -f "$LOCK_FILE" ]] && rm -f "$LOCK_FILE"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# 检查脚本是否已在运行
|
||||
if [[ -f "$LOCK_FILE" ]]; then
|
||||
if kill -0 $(cat "$LOCK_FILE") 2>/dev/null; then
|
||||
log "INFO" "脚本已在运行 (PID: $(cat "$LOCK_FILE"))"
|
||||
exit 1
|
||||
else
|
||||
log "WARN" "删除过期的锁定文件"
|
||||
rm -f "$LOCK_FILE"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Create lock file
|
||||
echo $$ > "$LOCK_FILE"
|
||||
|
||||
# 清理函数
|
||||
cleanup() {
|
||||
[[ -f "$LOCK_FILE" ]] && rm -f "$LOCK_FILE"
|
||||
log "INFO" "脚本执行完成"
|
||||
}
|
||||
|
||||
trap cleanup EXIT
|
||||
|
||||
# Database functions
|
||||
execute_sql() {
|
||||
local sql="$1"
|
||||
local result
|
||||
|
||||
result=$(mysql -h"$DB_HOST" -P"$DB_PORT" -u"$DB_USER" -p"$DB_PASS" "$DB_NAME" -e "$sql" 2>&1)
|
||||
local exit_code=$?
|
||||
|
||||
if [[ $exit_code -ne 0 ]]; then
|
||||
log "ERROR" "SQL执行失败: $sql"
|
||||
log "ERROR" "MySQL错误: $result"
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo "$result"
|
||||
}
|
||||
|
||||
# Process student ID card refresh in batch
|
||||
process_id_card_refresh_batch() {
|
||||
local batch_offset="$1"
|
||||
local batch_size="$2"
|
||||
|
||||
log "INFO" "开始处理学籍号刷新批次 - 偏移量: $batch_offset, 批次大小: $batch_size"
|
||||
|
||||
# 查找需要刷新的学生记录(ext_field2为空或空字符串,且id_card不为空)
|
||||
local find_records_sql="
|
||||
SELECT id, student_no, student_name, id_card, ext_field2
|
||||
FROM xg_student_msg
|
||||
WHERE (ext_field2 IS NULL OR ext_field2 = '')
|
||||
AND id_card IS NOT NULL
|
||||
AND id_card != ''
|
||||
AND deleted = 0
|
||||
ORDER BY id ASC
|
||||
LIMIT $batch_offset, $batch_size;
|
||||
"
|
||||
|
||||
local records
|
||||
records=$(execute_sql "$find_records_sql")
|
||||
|
||||
# 检查是否有记录需要处理
|
||||
local data_records
|
||||
data_records=$(echo "$records" | grep -E '^[0-9]' || true)
|
||||
|
||||
if [[ -z "$data_records" ]]; then
|
||||
log "INFO" "未找到需要刷新的学生记录"
|
||||
return 0
|
||||
fi
|
||||
|
||||
log "INFO" "找到需要刷新的学生记录,开始处理..."
|
||||
|
||||
local processed_count=0
|
||||
local failed_count=0
|
||||
local student_ids=""
|
||||
|
||||
# 收集需要更新的学生ID
|
||||
while IFS=$'\t' read -r student_id student_no student_name id_card current_ext_field2; do
|
||||
# Skip empty lines
|
||||
[[ -z "$student_id" ]] && continue
|
||||
|
||||
# 生成新的学籍号:'G' + id_card
|
||||
local new_id_card="'G$id_card'"
|
||||
|
||||
# 构建学生ID列表,用于批量更新
|
||||
if [[ -z "$student_ids" ]]; then
|
||||
student_ids="$student_id"
|
||||
else
|
||||
student_ids="$student_ids,$student_id"
|
||||
fi
|
||||
|
||||
log "INFO" "准备刷新学籍号 - 学生ID: $student_id, 学号: $student_no, 姓名: $student_name, 身份证: $id_card, 新学籍号: G$id_card"
|
||||
|
||||
((processed_count++))
|
||||
|
||||
# 单个更新(为了记录详细日志)
|
||||
local update_sql="
|
||||
UPDATE xg_student_msg SET
|
||||
ext_field2 = 'G$id_card',
|
||||
update_by = 1,
|
||||
update_time = NOW()
|
||||
WHERE id = $student_id AND deleted = 0;
|
||||
"
|
||||
|
||||
if execute_sql "$update_sql"; then
|
||||
log "INFO" "成功刷新学籍号 - 学生ID: $student_id, 学号: $student_no, 新学籍号: G$id_card"
|
||||
else
|
||||
log "ERROR" "刷新学籍号失败 - 学生ID: $student_id, 学号: $student_no"
|
||||
((failed_count++))
|
||||
fi
|
||||
|
||||
done <<< "$data_records"
|
||||
|
||||
log "INFO" "批次处理完成 - 找到记录: $processed_count, 失败: $failed_count"
|
||||
|
||||
# 返回成功处理的记录数
|
||||
echo $((processed_count - failed_count))
|
||||
}
|
||||
|
||||
# 一次性刷新所有符合条件的学籍号
|
||||
process_all_id_card_refresh() {
|
||||
log "INFO" "开始一次性刷新所有学籍号"
|
||||
|
||||
# 直接执行批量更新,并获取受影响的行数
|
||||
local update_sql="
|
||||
UPDATE xg_student_msg SET
|
||||
ext_field2 = CONCAT('G', id_card),
|
||||
update_by = 1,
|
||||
update_time = NOW()
|
||||
WHERE (ext_field2 IS NULL OR ext_field2 = '')
|
||||
AND id_card IS NOT NULL
|
||||
AND id_card != ''
|
||||
AND deleted = 0;
|
||||
|
||||
SELECT ROW_COUNT() as affected_rows;
|
||||
"
|
||||
|
||||
local update_result
|
||||
update_result=$(execute_sql "$update_sql")
|
||||
|
||||
if [[ $? -eq 0 ]]; then
|
||||
# 从结果中提取受影响的行数
|
||||
local affected_rows
|
||||
affected_rows=$(echo "$update_result" | grep -E '^[0-9]' | awk '{print $1}' | tail -1)
|
||||
|
||||
if [[ "$affected_rows" -eq 0 ]]; then
|
||||
log "INFO" "没有需要刷新的学籍号记录"
|
||||
else
|
||||
log "INFO" "学籍号刷新完成 - 成功更新: $affected_rows 条记录"
|
||||
fi
|
||||
return 0
|
||||
else
|
||||
log "ERROR" "学籍号刷新失败"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Main function
|
||||
main() {
|
||||
log "INFO" "启动学生学籍号刷新处理脚本"
|
||||
log "INFO" "处理类型: ID_CARD_REFRESH(学籍号刷新)"
|
||||
log "INFO" "刷新规则: ext_field2 = 'G' + id_card"
|
||||
|
||||
# 测试数据库连接
|
||||
log "INFO" "正在测试数据库连接..."
|
||||
if ! execute_sql "SELECT 1;" >/dev/null 2>&1; then
|
||||
error_exit "数据库连接失败"
|
||||
fi
|
||||
log "INFO" "数据库连接成功"
|
||||
|
||||
# 检查命令行参数
|
||||
if [[ $# -gt 0 && "$1" == "once" ]]; then
|
||||
log "INFO" "执行一次性刷新模式"
|
||||
if process_all_id_card_refresh; then
|
||||
log "INFO" "一次性刷新完成"
|
||||
else
|
||||
error_exit "一次性刷新失败"
|
||||
fi
|
||||
else
|
||||
log "INFO" "执行循环监控模式,间隔10分钟"
|
||||
|
||||
# Main processing loop
|
||||
while true; do
|
||||
log "INFO" "开始新的处理周期"
|
||||
|
||||
# 直接执行批量刷新,如果有记录需要处理就会更新,没有则不影响
|
||||
if process_all_id_card_refresh; then
|
||||
log "INFO" "本轮处理周期完成"
|
||||
else
|
||||
log "WARN" "本轮处理周期出现错误,将继续下一轮"
|
||||
fi
|
||||
|
||||
# 等待配置的间隔时间后开始下一个周期
|
||||
log "INFO" "等待 $CYCLE_INTERVAL 秒后开始下一个处理周期..."
|
||||
sleep $CYCLE_INTERVAL
|
||||
done
|
||||
fi
|
||||
}
|
||||
|
||||
# Run main function
|
||||
main "$@"
|
||||
Executable
+58
@@ -0,0 +1,58 @@
|
||||
#!/bin/bash
|
||||
|
||||
# 学籍号刷新快速执行脚本
|
||||
# 用法: ./run_refresh.sh [once|monitor|bg]
|
||||
# 参数说明:
|
||||
# once - 一次性刷新所有学籍号(默认)
|
||||
# monitor- 持续监控模式,前台执行
|
||||
# bg - 持续监控模式,后台执行
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
SCRIPT_FILE="${SCRIPT_DIR}/process_student_id_card_refresh.sh"
|
||||
|
||||
if [[ ! -f "$SCRIPT_FILE" ]]; then
|
||||
echo "错误:主脚本文件 $SCRIPT_FILE 未找到!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 检查参数
|
||||
MODE="${1:-once}"
|
||||
|
||||
echo "==============================================="
|
||||
echo "学生学籍号刷新脚本"
|
||||
echo "执行模式: $MODE"
|
||||
echo "时间: $(date '+%Y-%m-%d %H:%M:%S')"
|
||||
|
||||
if [[ "$MODE" == "bg" ]]; then
|
||||
echo "运行方式: 后台守护进程"
|
||||
else
|
||||
echo "运行方式: $([[ "$MODE" == "monitor" ]] && echo "前台守护进程" || echo "一次性执行")"
|
||||
fi
|
||||
|
||||
echo "==============================================="
|
||||
|
||||
case "$MODE" in
|
||||
"once")
|
||||
echo "执行一次性刷新模式..."
|
||||
"$SCRIPT_FILE" once
|
||||
;;
|
||||
"monitor")
|
||||
echo "执行持续监控模式(前台)..."
|
||||
"$SCRIPT_FILE"
|
||||
;;
|
||||
"bg")
|
||||
echo "执行持续监控模式(后台)..."
|
||||
echo "日志文件: ${SCRIPT_DIR}/student_id_card_refresh/process_student_id_card_refresh_$(date +%Y%m%d)_*.log"
|
||||
nohup "$SCRIPT_FILE" > /dev/null 2>&1 &
|
||||
echo "脚本已启动后台运行,PID: $!"
|
||||
echo "使用 'ps aux | grep process_student_id_card_refresh' 查看运行状态"
|
||||
echo "使用 'pkill -f process_student_id_card_refresh' 停止脚本"
|
||||
;;
|
||||
*)
|
||||
echo "用法: $0 [once|monitor|bg]"
|
||||
echo " once - 一次性刷新所有学籍号(默认)"
|
||||
echo " monitor- 持续监控模式,前台执行"
|
||||
echo " bg - 持续监控模式,后台执行(推荐)"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
@@ -0,0 +1,331 @@
|
||||
# 学生学籍号刷新脚本设计说明
|
||||
|
||||
## 📋 文档说明
|
||||
|
||||
**版本**: v1.0.0
|
||||
**制定时间**: 2025-12-26
|
||||
**适用范围**: 学生学籍号自动刷新业务
|
||||
**重要性**: ⚠️ **核心数据处理脚本** - 自动刷新学生学籍号信息
|
||||
|
||||
---
|
||||
|
||||
## 🎯 设计目标
|
||||
|
||||
### 1. 业务需求
|
||||
|
||||
- **自动刷新学籍号**: 自动为学生生成学籍号('G' + 身份证号)
|
||||
- **数据完整性**: 确保所有学生的ext_field2字段都有正确的学籍号
|
||||
- **批量处理**: 支持大量数据的批量处理
|
||||
- **监控守护**: 持续监控并处理新的学生数据
|
||||
|
||||
### 2. 技术要求
|
||||
|
||||
- **数据安全性**: 遵循项目数据库设计规范
|
||||
- **性能优化**: 合理的批量处理策略,支持高效更新
|
||||
- **扩展性**: 支持一次性处理和持续监控两种模式
|
||||
- **容错性**: 完善的错误处理和重试机制
|
||||
|
||||
---
|
||||
|
||||
## 🗄️ 脚本架构
|
||||
|
||||
### 文件结构
|
||||
|
||||
```
|
||||
学籍号刷新/
|
||||
├── data.conf # 开发环境配置
|
||||
├── data.conf.prod # 生产环境配置
|
||||
├── process_student_id_card_refresh.sh # 主处理脚本
|
||||
└── 学生学籍号刷新脚本设计说明.md # 设计文档
|
||||
```
|
||||
|
||||
### 脚本模式
|
||||
|
||||
#### 1. 一次性处理模式
|
||||
```bash
|
||||
./process_student_id_card_refresh.sh once
|
||||
```
|
||||
- 一次性刷新所有符合条件的学籍号
|
||||
- 适合初始化或手动触发处理
|
||||
|
||||
#### 2. 持续监控模式(默认)
|
||||
```bash
|
||||
./process_student_id_card_refresh.sh
|
||||
```
|
||||
- 每10分钟检查一次需要刷新的记录
|
||||
- 持续运行的守护进程模式
|
||||
|
||||
---
|
||||
|
||||
## 🔄 处理逻辑
|
||||
|
||||
### 刷新规则
|
||||
|
||||
**学籍号生成规则**: `ext_field2 = 'G' + id_card`
|
||||
|
||||
### 处理条件
|
||||
|
||||
```sql
|
||||
WHERE (ext_field2 IS NULL OR ext_field2 = '')
|
||||
AND id_card IS NOT NULL
|
||||
AND id_card != ''
|
||||
AND deleted = 0
|
||||
```
|
||||
|
||||
### 更新语句
|
||||
|
||||
```sql
|
||||
UPDATE xg_student_msg SET
|
||||
ext_field2 = CONCAT('G', id_card),
|
||||
update_by = 1,
|
||||
update_time = NOW()
|
||||
WHERE (ext_field2 IS NULL OR ext_field2 = '')
|
||||
AND id_card IS NOT NULL
|
||||
AND id_card != ''
|
||||
AND deleted = 0;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ 配置说明
|
||||
|
||||
### data.conf 配置参数
|
||||
|
||||
```bash
|
||||
# Database configuration
|
||||
DB_HOST="47.99.164.66" # 数据库主机
|
||||
DB_PORT="3306" # 数据库端口
|
||||
DB_NAME="hzdyjs_xg" # 数据库名
|
||||
DB_USER="root" # 数据库用户
|
||||
DB_PASS="hn02le.34lkdLKD" # 数据库密码
|
||||
|
||||
# Script configuration
|
||||
LOG_DIR="${SCRIPT_DIR}/student_id_card_refresh" # 日志目录
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # 脚本目录
|
||||
LOCK_FILE="${SCRIPT_DIR}/script.lock" # 锁定文件
|
||||
|
||||
# Processing configuration
|
||||
BATCH_SIZE=500 # 批次处理大小
|
||||
MAX_RETRY=3 # 最大重试次数
|
||||
RETRY_DELAY=5 # 重试间隔(秒)
|
||||
CYCLE_INTERVAL=600 # 循环间隔(秒,默认10分钟)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 使用方法
|
||||
|
||||
### 1. 准备工作
|
||||
|
||||
```bash
|
||||
# 进入脚本目录
|
||||
cd 学籍号刷新
|
||||
|
||||
# 给脚本添加执行权限
|
||||
chmod +x process_student_id_card_refresh.sh
|
||||
```
|
||||
|
||||
### 2. 一次性处理模式
|
||||
|
||||
```bash
|
||||
# 执行一次性刷新
|
||||
./process_student_id_card_refresh.sh once
|
||||
```
|
||||
|
||||
### 3. 持续监控模式
|
||||
|
||||
```bash
|
||||
# 使用快速启动脚本(推荐)
|
||||
./run_refresh.sh bg
|
||||
|
||||
# 或直接后台运行
|
||||
nohup ./process_student_id_card_refresh.sh > /dev/null 2>&1 &
|
||||
```
|
||||
|
||||
### 4. 停止脚本
|
||||
|
||||
```bash
|
||||
# 查找进程
|
||||
ps aux | grep process_student_id_card_refresh
|
||||
|
||||
# 终止进程
|
||||
kill -9 <PID>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 日志和监控
|
||||
|
||||
### 日志文件
|
||||
|
||||
日志文件保存在 `student_id_card_refresh/` 目录下:
|
||||
- 文件命名: `process_student_id_card_refresh_YYYYMMDD_HHMMSS.log`
|
||||
- 包含详细的处理过程和错误信息
|
||||
|
||||
### 日志内容
|
||||
|
||||
```
|
||||
[2025-12-26 14:30:15] [INFO] 启动学生学籍号刷新处理脚本
|
||||
[2025-12-26 14:30:15] [INFO] 处理类型: ID_CARD_REFRESH(学籍号刷新)
|
||||
[2025-12-26 14:30:15] [INFO] 数据库连接成功
|
||||
[2025-12-26 14:30:15] [INFO] 发现 1250 条需要刷新的学籍号记录,开始处理...
|
||||
[2025-12-26 14:30:20] [INFO] 批次处理完成 - 找到记录: 500, 失败: 0
|
||||
```
|
||||
|
||||
### 监控要点
|
||||
|
||||
1. **处理进度**: 定期检查日志中的处理进度
|
||||
2. **错误统计**: 关注失败的记录数和错误原因
|
||||
3. **性能监控**: 监控数据库负载和脚本运行状态
|
||||
|
||||
---
|
||||
|
||||
## 🔒 安全考虑
|
||||
|
||||
### 数据安全
|
||||
|
||||
1. **多租户隔离**: 通过deleted字段和业务逻辑隔离
|
||||
2. **事务安全**: 每次更新都有明确的条件限制
|
||||
3. **备份机制**: 更新前建议备份相关数据
|
||||
4. **权限控制**: 脚本需要数据库写入权限
|
||||
|
||||
### 运行安全
|
||||
|
||||
1. **单实例运行**: 通过锁定文件防止多实例同时运行
|
||||
2. **错误处理**: 完善的错误捕获和日志记录
|
||||
3. **资源控制**: 合理的批次大小和处理间隔
|
||||
|
||||
---
|
||||
|
||||
## 🚀 性能优化
|
||||
|
||||
### 批量处理策略
|
||||
|
||||
- **批次大小**: 500条/批,避免单次处理过多数据
|
||||
- **处理间隔**: 10分钟,与学生换班级脚本保持一致
|
||||
- **并发控制**: 单进程运行,避免数据库压力
|
||||
|
||||
### 数据库优化
|
||||
|
||||
```sql
|
||||
-- 建议添加的索引(如果性能需要)
|
||||
ALTER TABLE xg_student_msg ADD INDEX idx_ext_field2_deleted (ext_field2, deleted);
|
||||
ALTER TABLE xg_student_msg ADD INDEX idx_id_card_deleted (id_card, deleted);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📞 维护说明
|
||||
|
||||
### 定期维护任务
|
||||
|
||||
1. **日志清理**: 定期清理历史日志文件
|
||||
2. **配置检查**: 检查配置文件是否正确
|
||||
3. **数据库监控**: 监控脚本对数据库的影响
|
||||
4. **性能调优**: 根据实际情况调整批次大小和间隔
|
||||
|
||||
### 故障处理
|
||||
|
||||
#### 常见问题
|
||||
|
||||
1. **数据库连接失败**
|
||||
- 检查网络连接和数据库状态
|
||||
- 验证配置文件中的数据库参数
|
||||
|
||||
2. **权限不足**
|
||||
- 检查数据库用户的权限
|
||||
- 确认用户有UPDATE权限
|
||||
|
||||
3. **磁盘空间不足**
|
||||
- 清理日志文件
|
||||
- 检查磁盘空间使用情况
|
||||
|
||||
#### 应急处理
|
||||
|
||||
```bash
|
||||
# 强制停止脚本
|
||||
pkill -f process_student_id_card_refresh
|
||||
|
||||
# 清理锁定文件
|
||||
rm -f process_student_id_card_refresh.lock
|
||||
|
||||
# 手动执行SQL验证
|
||||
mysql -h host -u user -p database -e "UPDATE语句"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📈 统计分析
|
||||
|
||||
### 处理统计
|
||||
|
||||
```sql
|
||||
-- 查看学籍号刷新情况
|
||||
SELECT
|
||||
COUNT(*) as total_students,
|
||||
SUM(CASE WHEN ext_field2 LIKE 'G%' THEN 1 ELSE 0 END) as with_id_card,
|
||||
SUM(CASE WHEN ext_field2 IS NULL OR ext_field2 = '' THEN 1 ELSE 0 END) as need_refresh
|
||||
FROM xg_student_msg
|
||||
WHERE deleted = 0;
|
||||
```
|
||||
|
||||
### 效果验证
|
||||
|
||||
```sql
|
||||
-- 验证学籍号格式是否正确
|
||||
SELECT
|
||||
student_no,
|
||||
student_name,
|
||||
id_card,
|
||||
ext_field2,
|
||||
CONCAT('G', id_card) as expected_id_card
|
||||
FROM xg_student_msg
|
||||
WHERE ext_field2 LIKE 'G%'
|
||||
AND CONCAT('G', id_card) != ext_field2
|
||||
AND deleted = 0;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ 重要提醒
|
||||
|
||||
1. **数据备份**: 执行脚本前务必备份数据库
|
||||
2. **测试验证**: 先在测试环境验证脚本功能
|
||||
3. **监控运行**: 持续监控脚本运行状态和日志
|
||||
4. **权限控制**: 确保脚本运行账号有适当权限
|
||||
5. **文档同步**: 脚本变更必须同步更新本文档
|
||||
|
||||
---
|
||||
|
||||
## 🔗 相关文档
|
||||
|
||||
- **数据库设计**: `docs/设计文档/database/mysql-database-standard.md`
|
||||
- **项目规范**: `.cursor/cursorrules-backend.md`
|
||||
- **API文档**: `docs/开发文档/API/用户认证登录API.md`
|
||||
|
||||
---
|
||||
|
||||
### 🔧 版本历史
|
||||
|
||||
#### v1.0.2 (2025-12-26) - 易用性增强版
|
||||
- ✅ **后台执行**: 新增`run_refresh.sh bg`模式,支持后台守护进程
|
||||
- ✅ **用户友好**: 提供PID显示、状态检查和停止命令提示
|
||||
- ✅ **使用简化**: 一键启动后台监控,无需手动nohup
|
||||
|
||||
#### v1.0.1 (2025-12-26) - 性能优化版
|
||||
- ✅ **逻辑简化**: 移除不必要的预查询统计,直接执行UPDATE语句
|
||||
- ✅ **效率提升**: 使用`SELECT ROW_COUNT()`获取受影响行数,一次查询完成所有操作
|
||||
- ✅ **代码精简**: 减少数据库查询次数,降低系统开销
|
||||
- ✅ **监控优化**: 监控模式也使用简化的处理逻辑
|
||||
|
||||
#### v1.0.0 (2025-12-26) - 初始版本
|
||||
- ✅ 基础学籍号刷新功能
|
||||
- ✅ 批量处理和监控模式
|
||||
- ✅ 完善的日志和错误处理
|
||||
|
||||
---
|
||||
|
||||
**最后更新**: 2025-12-26
|
||||
**当前版本**: v1.0.2
|
||||
**维护人员**: AI Assistant
|
||||
**联系方式**: 项目技术团队
|
||||
Reference in new issue
Block a user