Category Archives: 技术分享

Mysql incorrect string value

这是由于mysql字段的存储字符集存储不了,比如utf8中存储不了一些表情符号,比如微信上的昵称:abc🍃,解决办法就是存储要修改对应的编码字符集,与数据库的通讯链接也要修改对应的编码字符集

  1. 把对应字段的字符集修改成utf8mb4
  2. 同时把代码中数据库链接驱动中的字符集也修改成utf8mb4,比如用pdo需要设置 :SET NAMES utf8mb4

两者少一个都不行

gitlab 备份、升级与迁移

备份

gitlab-rake gitlab:backup:create

如果是putty远程连接服务器执行该命令,可能由于某些git库太大导致putty迟迟没有反应而导致链接中断,链接中断后该命令也会终止执行,所以可以通过nohup 来把命令脱离命令行执行:

nohup gitlab-rake gitlab:backup:create 2>&1 &

通过在nohup.out 中可以查看输出日志

备份成功后在/var/opt/gitlab/backups中会产生一个类似于1610804990_gitlab_backup.tar的文件,一定要确保backup完成在copy该文件,上面的nohup.out中最后几行是下面情况时,说明备份完成了:

done
Dumping uploads …
done
Creating backup archive: 1610804990_gitlab_backup.tar … done
Uploading backup archive to remote storage … skipped
Deleting tmp directories … rake aborted!
Don’t know how to build task ‘/var/opt/backlog.log’

(See full trace by running task with –trace)
done
Deleting old backups … skipping

恢复

gitlab-rake gitlab:backup:restore BACKUP=/var/opt/gitlab/backups/1610804990

注意backup中的文件名不是完整的1610804990_gitlab_backup.tar,而只是_gitlab_backup.tar的前面部分

迁移

可以用scp把备份文件上传到目标服务器,在目标服务器上执行:scp src_username@src_ip:/var/opt/gitlab/backups/1481529483_gitlab_backup.tar /var/opt/gitlab/backups

升级

gitlab升级不能一次性跨大版本升级,比如不能6.x升级到13.x,只能在一个major版本升到最高级别后在升级到下一个major版本(版本号格式:major.minor),官方给出来的升级顺序是:

https://docs.gitlab.com/ee/update/#upgrade-paths

但我从7.6.x 升级到13.7并没有完全按照这个顺序,基本规则是升级到当前major版本的最后一个版本后,再升级到下一个major的第一个版本,一次类推直到最新版本

每个版本的镜像可以从这里下载:https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/

每次升级前,先stop服务

sudo gitlab-ctl stop unicorn
sudo gitlab-ctl stop sidekiq
sudo gitlab-ctl stop nginx
数据库不要stop,在安装时是需要备份数据库的,所以不能停

wget 下载好对应的rpm包后执行 rpm -Uvh 安装包

安装包会备份数据库部分,安装新版本,删除旧版本,升级成功收gitlab-ctl restart即可

全新安装LAMP环境

安装apache

1.使用yum安装apache:

yum install -y httpd

2.下载完成后启动:

systemctl start httpd.service
systemctl status httpd.service(查看运行状态)

3.设置apache开机自启动

systemclt enable httpd.service

至此apache安装完毕

安装mod_ssl

yum -y install mod_ssl

安装mariadb

1.首先设置数据源

curl -sS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup | sudo bash

2.更新缓存

yum clean all  
yum makecache  
yum repolist

3.显示可安装的版本

#这个可以看版本号
yum search mariadb –showduplicates 

yum search mariadb

4.安装

sudo yum install MariaDB-server galera-4 MariaDB-client MariaDB-shared MariaDB-backup MariaDB-common

5.启动服务

systemctl enable mariadb –now

6.配置数据库

mysql_secure_installation

首先是设置密码,会提示先输入密码

Enter current password for root (enter for none):<–初次运行直接回车

设置密码

Set root password? [Y/n] <– 是否设置root用户密码,输入y并回车或直接回车
New password: <– 设置root用户的密码
Re-enter new password: <– 再输入一次你设置的密码

其他配置

Remove anonymous users? [Y/n] <– 是否删除匿名用户,回车

Disallow root login remotely? [Y/n] <–是否禁止root远程登录,回车,

Remove test database and access to it? [Y/n] <– 是否删除test数据库,回车

Reload privilege tables now? [Y/n] <– 是否重新加载权限表,回车

初始化MariaDB完成,接下来测试登录

mysql -uroot -ppassword

设置数据库允许远程连接:

mysql -uroot -p(进入数据库)

查看MySQL库中的user表(user表中存着链接信息)

select host,user from user;

使用更新语句是root用户可以在任意IP的电脑上登录

update user set host=’%’,user=’root’ limit 1;

使修改生效

flush privileges;

退出Mariadb后并重启 mariadb服务

systemctl restart mariadb

至此,数据库安装完毕

安装php

1.首先安装 EPEL 源:

yum install epel-release

2.安装 REMI 源:

yum install http://rpms.remirepo.net/enterprise/remi-release-7.rpm

3.安装 Yum 源管理工具:

yum install yum-utils

4.安装 PHP7.3:

yum install -y php73-php-fpm php73-php-cli php73-php-bcmath php73-php-gd php73-php-json php73-php-mbstring php73-php-mcrypt php73-php-mysqlnd php73-php-opcache php73-php-pdo php73-php-pecl-crypto php73-php-pecl-mcrypt php73-php-pecl-geoip php73-php-recode php73-php-snmp php73-php-soap php73-php-xmll

5.启动php:

systemctl enable php73-php-fpm(开机自启动)
systemctl start php73-php-fpm(启动)

配置apache虚拟机

在/etc/httpd/conf.d文件夹新建一个conf配置文件,再把以下内容拷贝到里面

例如:

  
    DocumentRoot “/var/www/html/web/app/public_html”   #项目根目录
    ServerName lottery.yidianhulian.com  #绑定的域名
       
      Options FollowSymLinks ExecCGI
      AllowOverride All
      Order allow,deny
      Allow from all
      Require all granted
    DirectoryIndex index.php index.html error/index.html
 
  ErrorDocument 400 /error/400.html
  ErrorDocument 403 /error/403.html
  ErrorDocument 404 /error/404.html
  ErrorDocument 500 /error/500.html
  ErrorDocument 501 /error/501.html
  ErrorDocument 502 /error/502.html
  ErrorDocument 503 /error/503.html
  ErrorDocument 504 /error/504.html
  ErrorDocument 505 /error/505.html
  ErrorDocument 506 /error/506.html
  ErrorDocument 507 /error/507.html
  ErrorDocument 510 /error/510.html

配置https

1.使用域名可以申请免费的ssl证书,选择apache版本,证书一共有三个文件,例如:

2.在服务器中新建一个文件夹,把这三个文件拷贝到里面

3.配置https其实就是配置/etc/httpd/conf.d/ssl.conf文件,使用yum下载mod_ssl后会自动生成这个文件。对文件进行配置,例如:

#
# When we also provide SSL we have to listen to the 
# the HTTPS port in addition.
#
Listen 443 https

##
##  SSL Global Context
##
##  All SSL configuration in this context applies both to
##  the main server and all SSL-enabled virtual hosts.
##

#   Pass Phrase Dialog:
#   Configure the pass phrase gathering process.
#   The filtering dialog program (`builtin’ is a internal
#   terminal dialog) has to provide the pass phrase on stdout.
SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog

#   Inter-Process Session Cache:
#   Configure the SSL Session Cache: First the mechanism 
#   to use and second the expiring timeout (in seconds).
SSLSessionCache         shmcb:/run/httpd/sslcache(512000)
SSLSessionCacheTimeout  300

#   Pseudo Random Number Generator (PRNG):
#   Configure one or more sources to seed the PRNG of the 
#   SSL library. The seed data should be of good random quality.
#   WARNING! On some platforms /dev/random blocks if not enough entropy
#   is available. This means you then cannot use the /dev/random device
#   because it would lead to very long connection times (as long as
#   it requires to make more entropy available). But usually those
#   platforms additionally provide a /dev/urandom device which doesn’t
#   block. So, if available, use this one instead. Read the mod_ssl User
#   Manual for more details.
SSLRandomSeed startup file:/dev/urandom  256
SSLRandomSeed connect builtin
#SSLRandomSeed startup file:/dev/random  512
#SSLRandomSeed connect file:/dev/random  512
#SSLRandomSeed connect file:/dev/urandom 512

#
# Use “SSLCryptoDevice” to enable any supported hardware
# accelerators. Use “openssl engine -v” to list supported
# engine names.  NOTE: If you enable an accelerator and the
# server does not start, consult the error logs and ensure
# your accelerator is functioning properly. 
#
SSLCryptoDevice builtin
#SSLCryptoDevice ubsec

##
## SSL Virtual Host Context
##

      #找到这个标签

# General setup for the virtual host, inherited from global configuration
ServerName lottery.yidianhulian.com   #配置域名
DocumentRoot /var/www/html/web/app/public_html  #配置项目根目录

  #添加这个标签允许访问该项目
 
      Options FollowSymLinks ExecCGI
      AllowOverride All
      Order allow,deny
      Allow from all
      Require all granted
    DirectoryIndex index.php index.html error/index.html
 

# Use separate log files for the SSL virtual host; note that LogLevel
# is not inherited from httpd.conf.
ErrorLog logs/ssl_error_log
TransferLog logs/ssl_access_log
LogLevel warn

#   SSL Engine Switch:
#   Enable/Disable SSL for this virtual host.
SSLEngine on

#   SSL Protocol support:
# List the enable protocol levels with which clients will be able to
# connect.  Disable SSLv2 access by default:
SSLProtocol all -SSLv2 -SSLv3  # 添加SSL协议支持协议,去掉不安全的协议。

#   SSL Cipher Suite:
#   List the ciphers that the client is permitted to negotiate.
#   See the mod_ssl documentation for a complete list.
SSLCipherSuite HIGH:!RC4:!MD5:!aNULL:!eNULL:!NULL:!DH:!EDH:!EXP:+MEDIUM   # 修改加密套件。

#   Speed-optimized SSL Cipher configuration:
#   If speed is your main concern (on busy HTTPS servers e.g.),
#   you might want to force clients to specific, performance
#   optimized ciphers. In this case, prepend those ciphers
#   to the SSLCipherSuite list, and enable SSLHonorCipherOrder.
#   Caveat: by giving precedence to RC4-SHA and AES128-SHA
#   (as in the example below), most connections will no longer
#   have perfect forward secrecy – if the server’s key is
#   compromised, captures of past or future traffic must be
#   considered compromised, too.
#SSLCipherSuite RC4-SHA:AES128-SHA:HIGH:MEDIUM:!aNULL:!MD5
SSLHonorCipherOrder on 

#   Server Certificate:
# Point SSLCertificateFile at a PEM encoded certificate.  If
# the certificate is encrypted, then you will be prompted for a
# pass phrase.  Note that a kill -HUP will prompt again.  A new
# certificate can be generated using the genkey(1) command.
#SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateFile /etc/httpd/cert/4520877_lottery.yidianhulian.com_public.crt  #证书所在目录。要一一对应,注意看文件名

#   Server Private Key:
#   If the key is not combined with the certificate, use this
#   directive to point at the key file.  Keep in mind that if
#   you’ve both a RSA and a DSA private key you can configure
#   both in parallel (to also allow the use of DSA ciphers, etc.)
SSLCertificateKeyFile /etc/httpd/cert/4520877_lottery.yidianhulian.com.key  #证书所在目录。要一一对应,注意看文件名

#   Server Certificate Chain:
#   Point SSLCertificateChainFile at a file containing the
#   concatenation of PEM encoded CA certificates which form the
#   certificate chain for the server certificate. Alternatively
#   the referenced file can be the same as SSLCertificateFile
#   when the CA certificates are directly appended to the server
#   certificate for convinience.
SSLCertificateChainFile /etc/httpd/cert/4520877_lottery.yidianhulian.com_chain.crt  #证书所在目录。要一一对应,注意看文件名

#   Certificate Authority (CA):
#   Set the CA certificate verification path where to find CA
#   certificates for client authentication or alternatively one
#   huge file containing all of them (file must be PEM encoded)
#SSLCACertificateFile /etc/pki/tls/certs/ca-bundle.crt

#   Client Authentication (Type):
#   Client certificate verification type and depth.  Types are
#   none, optional, require and optional_no_ca.  Depth is a
#   number which specifies how deeply to verify the certificate
#   issuer chain before deciding the certificate is not valid.
#SSLVerifyClient require
#SSLVerifyDepth  10

#   Access Control:
#   With SSLRequire you can do per-directory access control based
#   on arbitrary complex boolean expressions containing server
#   variable checks and other lookup directives.  The syntax is a
#   mixture between C and Perl.  See the mod_ssl documentation
#   for more details.  
#
#SSLRequire (    %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
#            and %{SSL_CLIENT_S_DN_O} eq “Snake Oil, Ltd.” \
#            and %{SSL_CLIENT_S_DN_OU} in {“Staff”, “CA”, “Dev”} \
#            and %{TIME_WDAY} >= 1 and %{TIME_WDAY} <= 5 \
#            and %{TIME_HOUR} >= 8 and %{TIME_HOUR} <= 20       ) \
#           or %{REMOTE_ADDR} =~ m/^192\.76\.162\.[0-9]+$/
#

#   SSL Engine Options:
#   Set various options for the SSL engine.
#   o FakeBasicAuth:
#     Translate the client X.509 into a Basic Authorisation.  This means that
#     the standard Auth/DBMAuth methods can be used for access control.  The
#     user name is the `one line’ version of the client’s X.509 certificate.
#     Note that no password is obtained from the user. Every entry in the user
#     file needs this password: `xxj31ZMTZzkVA’.
#   o ExportCertData:
#     This exports two additional environment variables: SSL_CLIENT_CERT and
#     SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
#     server (always existing) and the client (only existing when client
#     authentication is used). This can be used to import the certificates
#     into CGI scripts.
#   o StdEnvVars:
#     This exports the standard SSL/TLS related `SSL_*’ environment variables.
#     Per default this exportation is switched off for performance reasons,
#     because the extraction step is an expensive operation and is usually
#     useless for serving static content. So one usually enables the
#     exportation for CGI and SSI requests only.
#   o StrictRequire:
#     This denies access when “SSLRequireSSL” or “SSLRequire” applied even
#     under a “Satisfy any” situation, i.e. when it applies access is denied
#     and no other module can change it.
#   o OptRenegotiate:
#     This enables optimized SSL connection renegotiation handling when SSL
#     directives are used in per-directory context. 
#SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire

    SSLOptions +StdEnvVars


    SSLOptions +StdEnvVars

#   SSL Protocol Adjustments:
#   The safe and default but still SSL/TLS standard compliant shutdown
#   approach is that mod_ssl sends the close notify alert but doesn’t wait for
#   the close notify alert from client. When you need a different shutdown
#   approach you can use one of the following variables:
#   o ssl-unclean-shutdown:
#     This forces an unclean shutdown when the connection is closed, i.e. no
#     SSL close notify alert is send or allowed to received.  This violates
#     the SSL/TLS standard but is needed for some brain-dead browsers. Use
#     this when you receive I/O errors because of the standard approach where
#     mod_ssl sends the close notify alert.
#   o ssl-accurate-shutdown:
#     This forces an accurate shutdown when the connection is closed, i.e. a
#     SSL close notify alert is send and mod_ssl waits for the close notify
#     alert of the client. This is 100% SSL/TLS standard compliant, but in
#     practice often causes hanging connections with brain-dead browsers. Use
#     this only for browsers where you know that their SSL implementation
#     works correctly. 
#   Notice: Most problems of broken clients are also related to the HTTP
#   keep-alive facility, so you usually additionally want to disable
#   keep-alive for those clients, too. Use variable “nokeepalive” for this.
#   Similarly, one has to force some clients to use HTTP/1.0 to workaround
#   their broken HTTP/1.1 implementation. Use variables “downgrade-1.0” and
#   “force-response-1.0” for this.
BrowserMatch “MSIE [2-5]” \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0

#   Per-Server Logging:
#   The home of a custom SSL log file. Use this when you want a
#   compact non-error SSL logfile on a virtual host basis.
CustomLog logs/ssl_request_log \
          “%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \”%r\” %b”

小程序真机报request:fail ssl hand shake error错误解决办法

错误现象

真机调试是控制台报request:fail ssl hand shake error错误,功能无法正常使用

问题排查

出现这个问题一定是SSL的证书没有配置好,证书不能自颁发,要用有公信力的机构来颁发,这可以在阿里云或者腾讯云上用免费的证书即可

检查自己的证书情况

  1. 打开这个网址 https://www.myssl.cn/tools/check-server-cert.html
  2. 输入域名进行检查
  3. 检查结果会分别呈现证书、根证书、中间证书的的情况,如果提示哪个证书找不到,就需要对应的排查下自己服务器上的生成设置情况即可

Apache

  1. SSLCertificateFile
  2. SSLCertificateKeyFile
  3. SSLCertificateChainFile

以上三个设置不能被注释,并一定要设置正确

其他服务器

一般证书颁发机构都提供不同服务器证书格式的下载,如果没有根证书,中间证书,可通过这个工具生成:https://www.myssl.cn/tools/downloadchain.html

Centos 7.6 云服务器全新安装LAMP手册

安装清单

  1. centos 7.6 64
  2. httpd 2.4
  3. php 7.2
  4. mariadb 10.3

挂载硬盘

1. 硬盘分期

fdisk -l 查看自己的服务器上是否有没有挂载的硬盘

Centos 7.6 云服务器全新安装LAMP手册

格式化硬盘 fdisk [未格式化的硬盘],如fdisk /dev/xvdb 回车

Centos 7.6 云服务器全新安装LAMP手册

按上图操作后,再次用fdisk -l 将看到你硬盘已经格式化

Centos 7.6 云服务器全新安装LAMP手册

2.格式化硬盘

用mkfs.ext3 [你的硬盘],如mkfs.ext3 /dev/xvdb1 进行格式化;格式可以采用mkfs.ext4等看具体的用途,详细介绍看这里:https://linux.die.net/man/8/mkfs

3.挂载硬盘

创建要硬盘要挂载的目录,mkdir /var/www/html

挂载到目录 mount /dev/xvdb1 /var/www/html

配置/etc/fstab,让系统启动自动挂载 在文件的末尾加上

/dev/xvdb1 /var/www/html ext3 defaults 0 0

4.reboot 重启

安装apache

  1. yum install httpd
  2. systemctl start httpd
  3. systemctl enable httpd

安装php

  1. rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
  2. rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
  3. 安装是用php72w代替php即可,如yum install php72w php72w-mysql

安装mariadb

删除已有的mysql或者低版本mariadb

  1. rpm -qa | grep mariadb
  2. yum remove 要删除的rpm

增加rpm库

vi /etc/yum.repos.d/MariaDB.repo

在该文件中添加以下内容保存:

[mariadb]name=MariaDB

baseurl=http://mirrors.aliyun.com/mariadb/yum/10.3/centos7-amd64

gpgkey= http://mirrors.aliyun.com/mariadb/yum/RPM-GPG-KEY-MariaDB

gpgcheck = 1

systemctl start mariadb

systemctl enable mariadb

接下来进行MariaDB的相关简单配置

mysql_secure_installation

docker配置lamp环境笔记

基本概念

容器container本质是一个运行在主机上的进程

  • 查看运行的容器docker ps 默认只看运行中的
  • 查看所有的容器docker ps -a
  • 删除容器 docker rm 容器id[或容器名]
  • 停止一个容器 docker stop 容器id

镜像image是容器的静态概念(可以看做容器的“安装包”)

  • 查看安装的镜像docker images
  • 删除镜像docker rmi 镜像名

仓库是存放镜像的地方,跟git一样的

docker配置lamp环境笔记

镜像是打包好的运行环境,比如centos,做好这个镜像后上传到镜像库,别人就可以直接pull下来运行使用了。

安装过程记录

  1. 拉取镜像 docker pull centos
  2. 运行容器 docker run -it -p 8080:80 centos /bin/bash 该命令总是会创建一个新容器,如果要启动已经存在的容器,用docker start 容器id
  3. -i 交互模式
  4. -t 分配一个伪终端
  5. -p 主机端口:容器内的端口
  6. centos镜像名,公共受信任的镜像可以直接用镜像名,个人镜像一般是镜像库用户名/镜像名,镜像库和git库类似,如果要拉取进行的某个版本标签,可以镜像名:[Tag名]
  7. /bin/bash启动镜像后执行的命令,这里也就是启动后打开终端,然后可以使用centos的yum安装需要的包,systemctl start 启动需要的服务
  8. 但在systemctl start时报错:Failed to get D-Bus connection: Operation not permitted
  9. 这是因为D-Bus没有启动,启动选项加上 –privileged -d,并把命令从/bin/bash 修改为init
  10. init就是/usr/sbin/init, 里面会自动启动D-Bus
  11. –privileged 赋予容器更多的权限,以便正常启动D-Bus
  12. -d是进入后台运行,init并不是控制终端,执行完成后它会一直停在那个地方,无法进行命令交互,所以-d让他脱离命令行,进入后台运行
  13. 然后docker exec -it [容器id] /bin/bash 进入容器的终端,接着就可以进行使用centos的各项命令了
  14. 需要安装的内容
  15. PHP
  16. Apache
  17. MySQL
  18. openssh-server 方便通过ssh登录容器,主要是方便用sftp scp这些上传下载代码
  19. 目录共享,我们可以把docker容器中apache的工作目录挂载到主机上(运行docker的机器)的目录
  20. 增加ssh端口映射,如果run后发现需要增加端口, 比如增加ssh端口,可以这么做
  21. docker commit [容器id] [起一个名字],先把当前的容器另存为一个镜像
  22. 然后在用run启动这个镜像,并带上你添加的端口 docker run -it -p 2222:22 -p 8080:80 –privileged -d [commit时的镜像名字] init
  23. 但这里要注意,之前的端口映射也要加上 , 这里的做法只是在之前的容器上继续启动,之前镜像中安装的内容也还在,但run命令的参数和之前是一样的,增加内容而已
  24. vim /etc/ssh/sshd_config
  25. 添加PermitRootLogin yes
  26. 记得docker exec -it [容器id] /bin/bash后设置一个root密码
  27. 在上面主机和容器目录共享,在run时加上参数 -v [主机的目录]:[容器中的目录],比如 -v /workspace/ydeditor:/var/www/html