β

搭建FastDFS集群-二

Arccode's blog 71 阅读

背景

上一篇 搭建FastDFS集群-一 描述了如何从源码构建 FastDFS 服务, 并通过脚本进行文件的 上传/下载/删除; 本文将详细描述如何安装 nginx扩展模块 ,并使用 http协议 下载文件; 同时使用 Java Client 方式执行文件的 上传/下载/删除/获取文件详情等.

FastDFS 架构

安装 fastdfs-nginx-module

全新安装nginx或在已安装的nginx上添加模块请看文章 源码安装nginx ,笔者已安装nginx, 就按照添加模块方式进行安装.

查看当前已安装模块

[root@Node nginx-1.12.1]# nginx -V
nginx version: nginx/1.12.1
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC)
configure arguments: --prefix=/root/devTools/nginx-1.12.1 --with-pcre=/root/workspaces/local/pcre-8.41 --with-zlib=/root/workspaces/local/zlib-1.2.8
cd ~/workspaces/github
git clone git@github:happyfish100/fastdfs-nginx-module.git
cd /workspaces/local/nginx-1.12.1
./configure --prefix=/root/devTools/nginx-1.12.1 --with-pcre=/root/workspaces/local/pcre-8.41 --with-zlib=/root/workspaces/local/zlib-1.2.8 --add-module=/root/workspaces/github/fastdfs-nginx-module/src/
make

注: 切记不要执行 make install 指令, 否则将覆盖之前的配置文件

接下来切换到 nginx 安装目录, 备份可执行的二进制文件, 然后复制源码下 ./objs 目录下的对应文件到安装目录即可, 如下:

cd /root/devTools/nginx-1.12.1/
mv nginx nginx.bak
cp /root/workspaces/local/nginx-1.12.1/objs/nginx ./

查看

[root@Node nginx-1.12.1]# nginx -V
nginx version: nginx/1.12.1
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC)
configure arguments: --prefix=/root/devTools/nginx-1.12.1 --with-pcre=/root/workspaces/local/pcre-8.41 --with-zlib=/root/workspaces/local/zlib-1.2.8 --add-module=/root/workspaces/github/fastdfs-nginx-module/src/

安装完成

配置

配置fastdfs-nginx-module

cd ~/workspaces/github/fastdfs-nginx-module/src
cd ~/workspaces/github/fastdfs-nginx-module/

按照如下进行配置,

# FastDFS tracker_server can ocur more than once, and tracker_server format is
# "host:port", host can be hostname or ip address
# valid only when load_fdfs_parameters_from_tracker is true
tracker_server=192.168.10.201:22122

# if the url / uri including the group name
# set to false when uri like /M00/00/00/xxx
# set to true when uri like ${group_name}/M00/00/00/xxx, such as group1/M00/xxx
# default value is false
url_have_group_name = true

# store_path#, based 0, if store_path0 not exists, it's value is base_path
# the paths must be exist
# must same as storage.conf
store_path0=/home/data/fastdfs/storage

复制其余配置,

cd ~/workspaces/github/fastdfs/conf
cp anti-steal.jpg http.conf mime.types /etc/fdfs/

配置nginx

vim ~/devTools/nginx-1.12.1/conf/nginx.conf

server 节点中增加如下 location 配置,

location /group1/M00/ {
ngx_fastdfs_module;
}

测试nginx

在浏览器中输入 http://192.168.10.201 , 能看到 Welcome to nginx! , 说明nginx工作正常.

部分情况可能会出现 403 Forbidden , 可能是如下原因:

测试fastdfs-nginx-module

按照上一篇 搭建FastDFS集群-一 介绍的内容, 上传文件; 可获得 fileId (group, visual disk, data directory, file name), 例如:

group1/M00/00/00/wKgKyVnmAaOAMTPXAABDgcE2i8g51.jpeg

拼接上schema, host, port即可构成一个url; 例如:

http://192.168.10.201/group1/M00/00/00/wKgKyVnmAaOAMTPXAABDgcE2i8g51.jpeg

在浏览器中访问该url可获得相关信息

至此, fastdfs-nginx-module 扩展模块安装完成; 接下来将介绍如何用 java 代码操纵 fastDFS 中的文件.

Java 客户端

导入依赖库:

<dependency>
<groupId>net.arccode</groupId>
<artifactId>fastdfs-client-java</artifactId>
<version>1.27.0</version>
</dependency>

fdfs_client.conf 配置如下:

connect_timeout = 30
network_timeout = 60
charset = UTF-8
tracker_server = 192.168.10.201:22122
package com.service_im;

import org.apache.commons.io.IOUtils;
import org.csource.common.NameValuePair;
import org.csource.fastdfs.*;
import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.InputStream;

/**
* fastDFS 功能测试
*
* @author http://arccode.net
* @since 2017-10-19
*/
public class FastDFSTest {

private static final Logger LOG = LoggerFactory.getLogger(FastDFSTest.class);

private static final String FDFS_CLIENT_FILENAME = "fdfs_client.conf";//配置文件名
private static final String BASE_PATH = "/Users/ac/temp/";//本地存储基本路径
private static TrackerClient trackerClient = null;
private static TrackerServer trackerServer = null;
private static StorageServer storageServer = null;
private static StorageClient1 storageClient = null;

String groupName = "group1";
String remotePath = "M00/00/00/wKgKyVnpmxyAP6bdAABDgcE2i8g49.jpeg";

@Before
public void before() throws Exception {
//初始化配置文件
String resourcePath = FastDFSTest.class.getClassLoader().getResource("").getPath();
String fdfsConfigPath = resourcePath + FDFS_CLIENT_FILENAME;
ClientGlobal.init(fdfsConfigPath);

LOG.info("{}", ClientGlobal.configInfo());

//连接FastDFS,创建tracker和storage
trackerClient = new TrackerClient();
trackerServer = trackerClient.getConnection();
storageClient = new StorageClient1(trackerServer, storageServer);

}

@Test
public void uploadFile() throws Exception {

LOG.info("uploadFile start...");

File file = new File("/Users/ac/Pictures/for_test/t1.jpeg");
String extName = "jpeg";

//设置文件信息
NameValuePair[] nameValues = new NameValuePair[1];
nameValues[0] = new NameValuePair("fileName", file.getName());
LOG.info("=================>>>>>>>>>" + file.getName());

//读取本地文件到缓存中
InputStream inputStream = new FileInputStream(file);
long fileSize = file.length();
byte[] buffer = new byte[(int) fileSize];
inputStream.read(buffer);

//上传文件
String fileId = storageClient.upload_file1(buffer, extName, nameValues);
if (fileId != null) {
//成功
LOG.info("=================>>>>>>>>>" + fileId);
}
}

@Test
public void downloadFile() throws Exception {
LOG.info("downloadFile start...");

FileInfo fi = storageClient.get_file_info(groupName, remotePath);
LOG.info(fi.toString());


//下载文件,并将文件放到缓冲区内
byte[] buffer = storageClient.download_file(groupName, remotePath);

//查询文件信息
String fileName = "";
NameValuePair[] nameValues = storageClient.get_metadata(groupName, remotePath);
if (nameValues != null) {
for (NameValuePair nameValue : nameValues) {
LOG.info("{} → {}", nameValue.getName(), nameValue.getValue());
if ("fileName".equals(nameValue.getName())) {
fileName = nameValue.getValue();
}
}

//写入到磁盘中
IOUtils.write(buffer, new FileOutputStream(BASE_PATH + fileName));
} else {
LOG.info("error: 文件不存在...");
}

}

@Test
public void removeFile() throws Exception {

LOG.info("removeFile start...");

int result = storageClient.delete_file(groupName, remotePath);
LOG.info(result == 0 ? "删除成功" : "删除失败:" + result);
}


}

待解决问题

fastdfs-nginx-module 高可用方案配置.

已解决, 解决方案如下:

nginx中server节点中的location的配置修改如下:

location /group1/M00/ {
ngx_fastdfs_module;
}

如果还是没有生效, 可尝试如下方法, 定能解决:

  1. 查看 nginx 错误日志, 根据提示进行修改
  2. 重新编译 nginx ; 示例如下
./configure --prefix=/home/devTools/nginx-1.12.2 --with-pcre=/home/installPackage/pcre-8.41 --with-zlib=/home/installPackage/zlib-1.2.8 --add-module=/home/workspaces/github/fastdfs-nginx-module/src

make

之后把 objs/nginx 复制到nginx的安装目录进行覆盖.

相关内容

FastDFS架构剖析

搭建FastDFS集群-一

搭建FastDFS集群-二

本文将详细描述如何安装nginx扩展模块,并使用http协议下载文件; 同时使用Java Client方式执行文件的 上传/下载/删除/获取文件详情等.
作者:Arccode's blog
原文地址:搭建FastDFS集群-二, 感谢原作者分享。

发表评论