SRE文档

深浅模式

1.部署clickhosue ​

1.1 安装 ​

1.2 创建数据库 ​

sql
#创建库
CREATE DATABASE IF NOT EXISTS nginxlogs ENGINE=Atomic;

#创建表
CREATE TABLE nginxlogs.nginx_access
(
    `timestamp` DateTime64(3, 'Asia/Shanghai'),
    `server_ip` String,
    `domain` String,
    `request_method` String,
    `status` Int32,
    `top_path` String,
    `path` String,
    `query` String,
    `protocol` String,
    `referer` String,
    `upstreamhost` String,
    `responsetime` Float32,
    `upstreamtime` Float32,
    `duration` Float32,
    `request_length` Int32,
    `response_length` Int32,
    `client_ip` String,
    `client_latitude` Float32,
    `client_longitude` Float32,
    `remote_user` String,
    `remote_ip` String,
    `xff` String,
    `client_city` String,
    `client_region` String,
    `client_country` String,
    `http_user_agent` String,
    `client_browser_family` String,
    `client_browser_major` String,
    `client_os_family` String,
    `client_os_major` String,
    `client_device_brand` String,
    `client_device_model` String,
    `createdtime` DateTime64(3, 'Asia/Shanghai')
)
ENGINE = MergeTree
PARTITION BY toYYYYMMDD(timestamp)
PRIMARY KEY (timestamp,
 server_ip,
 status,
 top_path,
 domain,
 upstreamhost,
 client_ip,
 remote_user,
 request_method,
 protocol,
 responsetime,
 upstreamtime,
 duration,
 request_length,
 response_length,
 path,
 referer,
 client_city,
 client_region,
 client_country,
 client_browser_family,
 client_browser_major,
 client_os_family,
 client_os_major,
 client_device_brand,
 client_device_model
)
TTL toDateTime(timestamp) + toIntervalDay(30)
SETTINGS index_granularity = 8192;
#创建库
CREATE DATABASE IF NOT EXISTS nginxlogs ENGINE=Atomic;

#创建表
CREATE TABLE nginxlogs.nginx_access
(
    `timestamp` DateTime64(3, 'Asia/Shanghai'),
    `server_ip` String,
    `domain` String,
    `request_method` String,
    `status` Int32,
    `top_path` String,
    `path` String,
    `query` String,
    `protocol` String,
    `referer` String,
    `upstreamhost` String,
    `responsetime` Float32,
    `upstreamtime` Float32,
    `duration` Float32,
    `request_length` Int32,
    `response_length` Int32,
    `client_ip` String,
    `client_latitude` Float32,
    `client_longitude` Float32,
    `remote_user` String,
    `remote_ip` String,
    `xff` String,
    `client_city` String,
    `client_region` String,
    `client_country` String,
    `http_user_agent` String,
    `client_browser_family` String,
    `client_browser_major` String,
    `client_os_family` String,
    `client_os_major` String,
    `client_device_brand` String,
    `client_device_model` String,
    `createdtime` DateTime64(3, 'Asia/Shanghai')
)
ENGINE = MergeTree
PARTITION BY toYYYYMMDD(timestamp)
PRIMARY KEY (timestamp,
 server_ip,
 status,
 top_path,
 domain,
 upstreamhost,
 client_ip,
 remote_user,
 request_method,
 protocol,
 responsetime,
 upstreamtime,
 duration,
 request_length,
 response_length,
 path,
 referer,
 client_city,
 client_region,
 client_country,
 client_browser_family,
 client_browser_major,
 client_os_family,
 client_os_major,
 client_device_brand,
 client_device_model
)
TTL toDateTime(timestamp) + toIntervalDay(30)
SETTINGS index_granularity = 8192;

1.3 创建用户 ​

xml
在user.xml中添加
<users>
		<root>
            <password>123456</password>
            <networks>
                <ip>::/0</ip>
            </networks>
            <profile>default</profile>
            <quota>default</quota>
        </root>
</users>
在user.xml中添加
<users>
		<root>
            <password>123456</password>
            <networks>
                <ip>::/0</ip>
            </networks>
            <profile>default</profile>
            <quota>default</quota>
        </root>
</users>

2.部署grafana ​

2.1 安装 ​

看grafana章节

2.2 安装clickhouse插件 ​

clickhouse 数据源插件有两种，

1.Altinity plugin for ClickHouse

2.ClickHouse

按照说明，官方的clickHouse插件只支持ClickHouse 22.7+版本，

ClickHouse 22版本以下，选择Altinity plugin for ClickHouse这个插件

1.图形安装 ​

  1. From the Connections page in the sidebar, select the Add new connection tab.
  2. Search for ClickHouse and click on the signed plugin by Grafana Labs:

Select the ClickHouse plugin on the connections page

Install the ClickHouse plugin

2.命令安装 ​

grafana-cli plugins install vertamedia-clickhouse-datasource
grafana-cli plugins install vertamedia-clickhouse-datasource

3.离线安装 ​

下载地址，https://grafana.com/grafana/plugins/

下载离线包，放到指定目录下（/var/lib/grafana/plugins）进行解压，重启服务即可

2.3 添加数据源 ​

image-20241030143529814

  • 填写数据库ip和端口，其他保持不变

image-20241030143621500

  • 填写用户名和密码，其他保持不变

image-20241030143742182

最后点击保存和测试

3.部署vector ​