1 部署es

(1) 安装es

docker run -d \
  --name es \
    -e "ES_JAVA_OPTS=-Xms1g -Xmx1g" \
    -e "discovery.type=single-node" \
    -v /mydata/es/data:/usr/local/elasticsearch/data \
    -v /mydata/es/plugins:/usr/local/elasticsearch/plugins \
    -v /mydata/es/logs:/usr/local/elasticsearch/logs \
    --privileged \
    --network es-net \
    -p 9200:9200 \
    -p 9300:9300 \
elasticsearch:7.12.1

es8以上需要关闭ssl

PUT _all/_settings
{
  "index.blocks.read_only_allow_delete": null
}

(2) 安装kibina

docker run -d --name kibana -e ELASTICSEARCH_HOSTS=http://es:9200 --network=es-net -p 5601:5601  kibana:7.12.1

cd config
vim kibana.yml

(3) 安装ik分词器(离线)

#找到数据卷的位置
docker volume inspect es-plugins

#将ik分词器放入数据卷的位置/var/lib/docker/volumes/es-plugins/_data

(4) 安装ik分词器(在线)

docker exec -it es bash

cd /usr/share/elasticsearch/bin

./elasticsearch-plugin  install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.12.1/elasticsearch-analysis-ik-7.12.1.zip

docker restart es

(5) ElasticSearch和MySQL对应关系

(6) 创建索引

{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  },
  "mappings": {
    "properties": {
      "id": {
        "type": "keyword"
      },
      "companyId": {
        "type": "keyword"
      },
      "companyName": {
        "analyzer": "ik_max_word",
        "search_analyzer": "ik_smart",
        "type": "text"
      },
      "name": {
        "analyzer": "ik_max_word",
        "search_analyzer": "ik_smart",
        "type": "text"
      },
      "users": {
        "index": false,
        "type": "text"
      },
      "tags": {
        "analyzer": "ik_max_word",
        "search_analyzer": "ik_smart",
        "type": "text"
      },
      "mt": {
        "type": "keyword"
      },
      "mtName": {
        "type": "keyword"
      },
      "st": {
        "type": "keyword"
      },
      "stName": {
        "type": "keyword"
      },
      "grade": {
        "type": "keyword"
      },
      "teachmode": {
        "type": "keyword"
      },
      "pic": {
        "index": false,
        "type": "text"
      },
      "description": {
        "analyzer": "ik_max_word",
        "search_analyzer": "ik_smart",
        "type": "text"
      },
      "createDate": {
        "format": "yyyy-MM-dd HH:mm:ss",
        "type": "date"
      },
      "status": {
        "type": "keyword"
      },
      "remark": {
        "index": false,
        "type": "text"
      },
      "charge": {
        "type": "keyword"
      },json
      "price": {
        "type": "scaled_float",
        "scaling_factor": 100
      },
      "originalPrice": {
        "type": "scaled_float",
        "scaling_factor": 100
      },
      "validDays": {
        "type": "integer"
      }
    }
  }
}

2 索引同步

(1) 实时性要求高

1.同步调用:在向MySQL写数据后远程调用搜索服务es的接口写入索引(代码耦合性高)

2.canal:基于MySQL数据库的binlog,同步MySQL数据到消息队列,es,其他数据库等(学习成本高)

(binlog:mysql的一个二进制文件,记录了对数据库更新的SQL语句)

(2) 实时性要求不高

1.MQ:向MySQL写数据的时候向MQ写入消息,搜索服务es监听MQ,收到消息后写入索引

(优势:代码解耦)

(需要处理消息可靠性:生产者投递成功,消息持久化,消费者消费成功三个方面,以及消息幂等性)

(实时日志分析平台ELK包括ElasticSearch,Kibana,Logstash)

2.Logstash:负责收集,解析和转换日志信息,可以实现mysql和es之间的数据同步

(优势:代码解耦,官方推荐)

(增加了学习成本和维护成本)

3.任务调度xxl-job: 向mysql写数据的时候记录修改记录,开启一个定时任务根据修改记录将数据同步到es