Compare commits

..

No commits in common. "main" and "temp" have entirely different histories.
main ... temp

21 changed files with 114 additions and 614 deletions

View File

@ -1,75 +0,0 @@
# This is a basic workflow to help you get started with Actions
name: aliyundriver-Build
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ main ]
pull_request:
branches: [ main ]
# schedule:
# - cron: "0 0 * * */3"
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
inputs:
logLevel:
description: 'Log level'
required: true
default: 'warning'
tags:
description: 'Test scenario tags'
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
buildx:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v2
-
name: Get current date
id: date
run: echo "::set-output name=today::$(date +'%Y-%m-%d')"
-
name: Set up QEMU
uses: docker/setup-qemu-action@v1
with:
platforms: all
-
name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
-
name: Available platforms
run: echo ${{ steps.buildx.outputs.platforms }}
-
name: Cache Docker layers
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
-
name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build and push
uses: docker/build-push-action@v2
with:
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache
tags: |
zx5253/webdav-aliyundriver:latest
zx5253/webdav-aliyundriver:${{ steps.date.outputs.today }}

View File

@ -7,5 +7,4 @@ RUN cd /tmp/code && mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.ski
FROM openjdk:11-jdk-oracle
COPY --from=maven /tmp/code/target/*.jar /webdav.jar
EXPOSE 8080
ENV JAVA_OPTS="-Xmx1g"
ENTRYPOINT ["sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /webdav.jar"]
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/webdav.jar"]

122
README.md
View File

@ -1,32 +1,7 @@
说明:[1.1.0版本](https://github.com/zxbu/webdav-aliyundriver/releases/tag/v1.1.0)支持阿里Teambition网盘的webdav协议后续的2.x版本仅支持阿里云盘不再维护Teambition网盘版本
- [webdav-aliyundriver](#webdav-aliyundriver)
- [如何使用](#如何使用)
- [Jar包运行](#jar包运行)
- [容器运行](#容器运行)
- [Docker-Compose](#docker-compose)
- [参数说明](#参数说明)
- [QQ群](#qq群)
- [新手教程](#新手教程)
- [群晖](#群晖)
- [Windows10](#windows10)
- [Linux](#linux)
- [Mac](#mac)
- [客户端兼容性](#客户端兼容性)
- [浏览器获取refreshToken方式](#浏览器获取refreshtoken方式)
- [功能说明](#功能说明)
- [支持的功能](#支持的功能)
- [暂不支持的功能](#暂不支持的功能)
- [已知问题](#已知问题)
- [TODO](#todo)
- [免责声明](#免责声明)
# webdav-aliyundriver
本项目实现了阿里云盘的webdav协议只需要简单的配置一下就可以让阿里云盘变身为webdav协议的文件服务器。
基于此你可以把阿里云盘挂载为Windows、Linux、Mac系统的磁盘可以通过NAS系统做文件管理或文件同步更多玩法等你挖掘
# 如何使用
支持refreshToken登录方式具体看参数说明
## Jar包运行
@ -37,39 +12,11 @@ java -jar webdav.jar --aliyundrive.refresh-token="your refreshToken"
```
## 容器运行
```bash
docker run -d --name=webdav-aliyundriver --restart=always -p 8080:8080 -v /etc/localtime:/etc/localtime -v /etc/aliyun-driver/:/etc/aliyun-driver/ -e TZ="Asia/Shanghai" -e ALIYUNDRIVE_REFRESH_TOKEN="your refreshToken" -e ALIYUNDRIVE_AUTH_PASSWORD="admin" -e JAVA_OPTS="-Xmx1g" zx5253/webdav-aliyundriver
docker run -d --name=webdav-aliyundriver --restart=always -p 8080:8080 -v /etc/localtime:/etc/localtime -v /etc/aliyun-driver/:/etc/aliyun-driver/ -e TZ="Asia/Shanghai" -e ALIYUNDRIVE_REFRESH_TOKEN="your refreshToken" zx5253/webdav-aliyundriver
# /etc/aliyun-driver/ 挂载卷自动维护了最新的refreshToken建议挂载
# ALIYUNDRIVE_AUTH_PASSWORD 是admin账户的密码建议修改
# JAVA_OPTS 可修改最大内存占用,比如 -e JAVA_OPTS="-Xmx512m" 表示最大内存限制为512m
```
## Docker-Compose
```yml
version: "3.0"
services:
webdav-aliyundriver:
image: zx5253/webdav-aliyundriver
container_name: aliyundriver
environment:
- TZ=Asia/Shanghai
- ALIYUNDRIVE_REFRESH_TOKEN=refreshToken
- ALIYUNDRIVE_AUTH_USER_NAME=admin
- ALIYUNDRIVE_AUTH_PASSWORD=admin
- JAVA_OPTS=-Xmx1g
volumes:
- /etc/aliyun-driver/:/etc/aliyun-driver/
ports:
- 6666:8080
restart: always
# “refreshToken”请根据下文说明自行获取。
# “ALIYUNDRIVE_AUTH_USER-NAME”和“ALIYUNDRIVE_AUTH_PASSWORD”为连接用户名和密码建议更改。
# “/etc/aliyun-driver/:/etc/aliyun-driver/”,可以把冒号前改为指定目录,比如“/homes/USER/docker/alidriver/:/etc/aliyun-driver/”。
# 删除了“/etc/localtime:/etc/localtime”如有需要同步时间请自行添加在environment下。
# 端口6666可自行按需更改此端口为WebDAV连接端口,8080为容器内配置端口修改请量力而为。
# 建议不要保留这些中文注释以防报错比如QNAP。
```
# 参数说明
```bash
@ -77,49 +24,9 @@ services:
阿里云盘的refreshToken获取方式见下文
--server.port
非必填服务器端口号默认为8080
--aliyundrive.auth.enable=true
是否开启WebDav账户验证默认开启
--aliyundrive.auth.user-name=admin
WebDav账户默认admin
--aliyundrive.auth.password=admin
WebDav密码默认admin
--aliyundrive.work-dir=/etc/aliyun-driver/
token挂载路径如果多开的话需修改此配置
```
# QQ群
> 群号已满789738128
> 二群群号已满979024890
> 三群群号已满212673498
> 四群群号已满752067171
> 五群群号555954095
# 新手教程
## 群晖
TODO
## Windows10
TODO
## Linux
TODO
## Mac
TODO
# 客户端兼容性
| 客户端 | 下载 | 上传 | 备注 |
| :-----| ----: | :----: | :----: |
| 群辉Cloud Sync | 可用 | 可用 | 使用单向同步非常稳定 |
| Rclone | 可用 | 可用 | 推荐,支持各个系统 |
| Mac原生 | 可用 | 可用 | |
| Windows原生 | 可用 | 有点小问题 | 不建议,适配有点问题,上传报错 |
| RaiDrive | 可用 | 可用 | Windows平台下建议用这个 |
> 群号789738128
# 浏览器获取refreshToken方式
1. 先通过浏览器建议chrome打开阿里云盘官网并登录https://www.aliyundrive.com/drive/
@ -135,27 +42,16 @@ TODO
4. 文件下载
5. 文件删除
6. 文件上传(支持大文件自动分批上传)
7. 支持超大文件上传官方限制30G
8. 支持WebDav权限校验默认账户密码admin/admin
9. 文件下载断点续传
10. Webdav下的流媒体播放等功能
## 暂不支持的功能
1. 移动文件到其他目录的同时,修改文件名。比如 /a.zip 移动到 /b/a1.zip是不支持的
2. 文件上传断点续传
3. 部分客户端兼容性不好
1. 权限校验
2. 移动文件到其他目录的同时,修改文件名。比如 /a.zip 移动到 /b/a1.zip是不支持的
3. 文件上传断点续传
4. 文件下载断点续传
5. 同级目录下文件数量不能超过10000个建议不超过100否则性能比较差
## 已知问题
1. 没有做文件sha1校验不保证上传文件的100%准确性(一般场景下,是没问题的)
2. 通过文件名和文件大小判断是否重复。也就是说如果一个文件即使发生了更新,但其大小没有任何改变,是不会自动上传的
3. 不支持文件名包含 `/` 字符
3. 超大文件上传存在问题具体上限还不清楚我自己实测5G+大小的文件无法顺利上传
## TODO
1. 支持更多登录方式(验证码、账号密码等)
# 免责声明
1. 本软件为免费开源项目,无任何形式的盈利行为。
2. 本软件服务于阿里云盘,旨在让阿里云盘功能更强大。如有侵权,请与我联系,会及时处理。
3. 本软件皆调用官方接口实现无任何“Hack”行为无破坏官方接口行为。
5. 本软件仅做流量转发,不拦截、存储、篡改任何用户数据。
6. 严禁使用本软件进行盈利、损坏官方、散落任何违法信息等行为。
7. 本软件不作任何稳定性的承诺,如因使用本软件导致的文件丢失、文件破坏等意外情况,均与本软件无关。
2. 支持权限校验

View File

@ -1,17 +0,0 @@
version: '3'
services:
# https://github.com/zxbu/webdav-aliyundriver
webdav-aliyundriver:
image: zx5253/webdav-aliyundriver
container_name: webdav-aliyundriver
restart: always
volumes:
- /etc/localtime:/etc/localtime
- ./docker/etc/aliyun-driver/:/etc/aliyun-driver/
ports:
- "8080:8080"
tty: true
environment:
- TZ=Asia/Shanghai
- ALIYUNDRIVE_REFRESH_TOKEN=<change me>
- ALIYUNDRIVE_AUTH_PASSWORD=<change me>

View File

@ -10,7 +10,7 @@
</parent>
<groupId>com.github.zxbu</groupId>
<artifactId>webdav-aliyundriver</artifactId>
<version>2.4.0</version>
<version>2.2.0</version>
<name>webdav</name>
<description>Demo project for Spring Boot</description>
@ -30,12 +30,6 @@
<version>3.14.9</version>
</dependency>
<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
<version>2.7.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>

View File

@ -8,13 +8,11 @@ import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.servlet.FilterRegistrationBean;
import org.springframework.boot.web.servlet.ServletRegistrationBean;
import org.springframework.context.annotation.Bean;
import org.springframework.scheduling.annotation.EnableScheduling;
import java.util.LinkedHashMap;
import java.util.Map;
@SpringBootApplication
@EnableScheduling
public class WebdavTeambitionApplication {
public static void main(String[] args) {

View File

@ -7,9 +7,7 @@ import okhttp3.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.util.Assert;
import org.springframework.util.StringUtils;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
@ -44,14 +42,7 @@ public class AliYunDriverClient {
@Override
public Request authenticate(Route route, Response response) throws IOException {
if (response.code() == 401 && response.body() != null && response.body().string().contains("AccessToken")) {
String refreshTokenResult;
try {
refreshTokenResult = post("https://websv.aliyundrive.com/token/refresh", Collections.singletonMap("refresh_token", readRefreshToken()));
} catch (Exception e) {
// 如果置换token失败先清空原token文件再尝试一次
deleteRefreshTokenFile();
refreshTokenResult = post("https://websv.aliyundrive.com/token/refresh", Collections.singletonMap("refresh_token", readRefreshToken()));
}
String refreshTokenResult = post("https://websv.aliyundrive.com/token/refresh", Collections.singletonMap("refresh_token", readRefreshToken()));
String accessToken = (String) JsonUtil.getJsonNodeValue(refreshTokenResult, "access_token");
String refreshToken = (String) JsonUtil.getJsonNodeValue(refreshTokenResult, "refresh_token");
Assert.hasLength(accessToken, "获取accessToken失败");
@ -94,32 +85,12 @@ public class AliYunDriverClient {
}
public Response download(String url, HttpServletRequest httpServletRequest, long size ) {
Request.Builder builder = new Request.Builder().header("referer", "https://www.aliyundrive.com/");
String range = httpServletRequest.getHeader("range");
if (range != null) {
// 如果range最后 >= size 则去掉
String[] split = range.split("-");
if (split.length == 2) {
String end = split[1];
if (Long.parseLong(end) >= size) {
range = range.substring(0, range.lastIndexOf('-') + 1);
}
}
builder.header("range", range);
}
String ifRange = httpServletRequest.getHeader("if-range");
if (ifRange != null) {
builder.header("if-range", ifRange);
}
Request request = builder.url(url).build();
public InputStream download(String url) {
Request request = new Request.Builder().url(url).build();
Response response = null;
try {
response = okHttpClient.newCall(request).execute();
return response;
return response.body().byteStream();
} catch (IOException e) {
throw new WebdavException(e);
}
@ -207,16 +178,6 @@ public class AliYunDriverClient {
return aliYunDriveProperties.getUrl() + url;
}
private void deleteRefreshTokenFile() {
String refreshTokenPath = aliYunDriveProperties.getWorkDir() + "refresh-token";
Path path = Paths.get(refreshTokenPath);
try {
Files.delete(path);
} catch (IOException e) {
e.printStackTrace();
}
}
private String readRefreshToken() {
String refreshTokenPath = aliYunDriveProperties.getWorkDir() + "refresh-token";
Path path = Paths.get(refreshTokenPath);

View File

@ -1,28 +0,0 @@
package com.github.zxbu.webdavteambition.config;
import com.github.zxbu.webdavteambition.model.result.TFile;
import com.github.zxbu.webdavteambition.store.AliYunDriverClientService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
@Component
public class AliYunDriverCronTask {
@Autowired
private AliYunDriverClientService aliYunDriverClientService;
/**
* 每隔5分钟请求一下接口保证token不过期
*/
@Scheduled(initialDelay = 30 * 1000, fixedDelay = 5 * 60 * 1000)
public void refreshToken() {
try {
TFile root = aliYunDriverClientService.getTFileByPath("/");
aliYunDriverClientService.getTFiles(root.getFile_id());
} catch (Exception e) {
// nothing
}
}
}

View File

@ -1,8 +1,6 @@
package com.github.zxbu.webdavteambition.config;
import org.apache.catalina.CredentialHandler;
import org.apache.catalina.authenticator.AuthenticatorBase;
import org.apache.catalina.authenticator.BasicAuthenticator;
import org.apache.catalina.authenticator.DigestAuthenticator;
import org.apache.catalina.realm.GenericPrincipal;
import org.apache.catalina.realm.MessageDigestCredentialHandler;
import org.apache.catalina.realm.RealmBase;
@ -21,7 +19,7 @@ import java.util.Collections;
@Component
@ConditionalOnProperty(prefix = "aliyundrive.auth", name = "enable", matchIfMissing = true)
public class AuthTomcatConfig implements WebServerFactoryCustomizer<ConfigurableServletWebServerFactory>, Ordered {
public class EmbeddedTomcatConfig implements WebServerFactoryCustomizer<ConfigurableServletWebServerFactory>, Ordered {
@Autowired
private AliYunDriveProperties aliYunDriveProperties;
@ -39,7 +37,7 @@ public class AuthTomcatConfig implements WebServerFactoryCustomizer<Configurable
if (aliYunDriveProperties.getAuth().getUserName().equals(username)) {
return aliYunDriveProperties.getAuth().getPassword();
}
return null;
return "";
}
@Override
@ -48,11 +46,11 @@ public class AuthTomcatConfig implements WebServerFactoryCustomizer<Configurable
}
};
CredentialHandler credentialHandler = new MessageDigestCredentialHandler();
MessageDigestCredentialHandler credentialHandler = new MessageDigestCredentialHandler();
realm.setCredentialHandler(credentialHandler);
context.setRealm(realm);
AuthenticatorBase digestAuthenticator = new BasicAuthenticator();
DigestAuthenticator digestAuthenticator = new DigestAuthenticator();
SecurityConstraint securityConstraint = new SecurityConstraint();
securityConstraint.setAuthConstraint(true);
securityConstraint.addAuthRole("**");
@ -69,4 +67,4 @@ public class AuthTomcatConfig implements WebServerFactoryCustomizer<Configurable
public int getOrder() {
return Ordered.LOWEST_PRECEDENCE;
}
}
}

View File

@ -1,31 +0,0 @@
package com.github.zxbu.webdavteambition.model;
public class DownloadRequest {
private String drive_id;
private String file_id;
private Integer expire_sec = 14400;
public String getDrive_id() {
return drive_id;
}
public void setDrive_id(String drive_id) {
this.drive_id = drive_id;
}
public String getFile_id() {
return file_id;
}
public void setFile_id(String file_id) {
this.file_id = file_id;
}
public Integer getExpire_sec() {
return expire_sec;
}
public void setExpire_sec(Integer expire_sec) {
this.expire_sec = expire_sec;
}
}

View File

@ -1,17 +1,17 @@
package com.github.zxbu.webdavteambition.model;
public class Page {
private String marker;
private int offset;
private int limit;
private String order_by;
private String order_direction;
public String getMarker() {
return marker;
public int getOffset() {
return offset;
}
public void setMarker(String marker) {
this.marker = marker;
public void setOffset(int offset) {
this.offset = offset;
}
public int getLimit() {

View File

@ -1,42 +0,0 @@
package com.github.zxbu.webdavteambition.model;
import java.util.List;
public class RefreshUploadUrlRequest {
private String drive_id;
private List<UploadPreRequest.PartInfo> part_info_list;
private String file_id;
private String upload_id;
public String getDrive_id() {
return drive_id;
}
public void setDrive_id(String drive_id) {
this.drive_id = drive_id;
}
public List<UploadPreRequest.PartInfo> getPart_info_list() {
return part_info_list;
}
public void setPart_info_list(List<UploadPreRequest.PartInfo> part_info_list) {
this.part_info_list = part_info_list;
}
public String getFile_id() {
return file_id;
}
public void setFile_id(String file_id) {
this.file_id = file_id;
}
public String getUpload_id() {
return upload_id;
}
public void setUpload_id(String upload_id) {
this.upload_id = upload_id;
}
}

View File

@ -4,7 +4,6 @@ import java.util.List;
public class TFileListResult<T> {
private List<T> items;
private String next_marker;
public List<T> getItems() {
return items;
@ -13,12 +12,4 @@ public class TFileListResult<T> {
public void setItems(List<T> items) {
this.items = items;
}
public String getNext_marker() {
return next_marker;
}
public void setNext_marker(String next_marker) {
this.next_marker = next_marker;
}
}

View File

@ -2,8 +2,6 @@ package com.github.zxbu.webdavteambition.store;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import com.github.zxbu.webdavteambition.client.AliYunDriverClient;
import com.github.zxbu.webdavteambition.model.*;
import com.github.zxbu.webdavteambition.model.result.TFile;
@ -11,21 +9,16 @@ import com.github.zxbu.webdavteambition.model.result.TFileListResult;
import com.github.zxbu.webdavteambition.model.result.UploadPreResult;
import com.github.zxbu.webdavteambition.util.JsonUtil;
import net.sf.webdav.exceptions.WebdavException;
import okhttp3.HttpUrl;
import okhttp3.Response;
import org.apache.tomcat.util.http.fileupload.IOUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.util.StringUtils;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.io.InputStream;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.TimeUnit;
@Service
public class AliYunDriverClientService {
@ -34,64 +27,44 @@ public class AliYunDriverClientService {
private static String rootPath = "/";
private static int chunkSize = 10485760; // 10MB
private TFile rootTFile = null;
private static Cache<String, Set<TFile>> tFilesCache = Caffeine.newBuilder()
.initialCapacity(128)
.maximumSize(1024)
.expireAfterWrite(1, TimeUnit.MINUTES)
.build();
private ThreadLocal<Map<String, Set<TFile>>> tFilesCache = new ThreadLocal<>();
private final AliYunDriverClient client;
@Autowired
private VirtualTFileService virtualTFileService;
public AliYunDriverClientService(AliYunDriverClient aliYunDriverClient) {
this.client = aliYunDriverClient;
AliYunDriverFileSystemStore.setBean(this);
}
public Set<TFile> getTFiles(String nodeId) {
Set<TFile> tFiles = tFilesCache.get(nodeId, key -> {
// 获取真实的文件列表
return getTFiles2(nodeId);
});
Set<TFile> all = new LinkedHashSet<>(tFiles);
// 获取上传中的文件列表
Collection<TFile> virtualTFiles = virtualTFileService.list(nodeId);
all.addAll(virtualTFiles);
return all;
}
private Set<TFile> getTFiles2(String nodeId) {
List<TFile> tFileList = fileListFromApi(nodeId, null, new ArrayList<>());
tFileList.sort(Comparator.comparing(TFile::getUpdated_at).reversed());
Set<TFile> tFileSets = new LinkedHashSet<>();
for (TFile tFile : tFileList) {
if (!tFileSets.add(tFile)) {
LOGGER.info("当前目录下{} 存在同名文件:{},文件大小:{}", nodeId, tFile.getName(), tFile.getSize());
Map<String, Set<TFile>> map = tFilesCache.get();
if (map == null) {
map = new ConcurrentHashMap<>();
tFilesCache.set(map);
}
return map.computeIfAbsent(nodeId, key -> {
FileListRequest listQuery = new FileListRequest();
listQuery.setOffset(0);
listQuery.setLimit(10000);
listQuery.setOrder_by("updated_at");
listQuery.setOrder_direction("DESC");
listQuery.setDrive_id(client.getDriveId());
listQuery.setParent_file_id(nodeId);
String json = client.post("/file/list", listQuery);
TFileListResult<TFile> tFileListResult = JsonUtil.readValue(json, new TypeReference<TFileListResult<TFile>>() {
});
List<TFile> tFileList = tFileListResult.getItems();
tFileList.sort(Comparator.comparing(TFile::getUpdated_at).reversed());
Set<TFile> tFileSets = new LinkedHashSet<>();
for (TFile tFile : tFileList) {
if (!tFileSets.add(tFile)) {
LOGGER.info("当前目录下{} 存在同名文件:{},文件大小:{}", nodeId, tFile.getName(), tFile.getSize());
}
}
}
// 对文件名进行去重只保留最新的一个
return tFileSets;
}
private List<TFile> fileListFromApi(String nodeId, String marker, List<TFile> all) {
FileListRequest listQuery = new FileListRequest();
listQuery.setMarker(marker);
listQuery.setLimit(100);
listQuery.setOrder_by("updated_at");
listQuery.setOrder_direction("DESC");
listQuery.setDrive_id(client.getDriveId());
listQuery.setParent_file_id(nodeId);
String json = client.post("/file/list", listQuery);
TFileListResult<TFile> tFileListResult = JsonUtil.readValue(json, new TypeReference<TFileListResult<TFile>>() {
// 对文件名进行去重只保留最新的一个
return tFileSets;
});
all.addAll(tFileListResult.getItems());
if (!StringUtils.hasLength(tFileListResult.getNext_marker())) {
return all;
}
return fileListFromApi(nodeId, tFileListResult.getNext_marker(), all);
}
@ -152,43 +125,20 @@ public class AliYunDriverClientService {
UploadPreResult uploadPreResult = JsonUtil.readValue(json, UploadPreResult.class);
List<UploadPreRequest.PartInfo> partInfoList = uploadPreResult.getPart_info_list();
if (partInfoList != null) {
if (size > 0) {
virtualTFileService.createTFile(parent.getFile_id(), uploadPreResult);
}
LOGGER.info("文件预处理成功,开始上传。文件名:{}上传URL数量{}", path, partInfoList.size());
byte[] buffer = new byte[chunkSize];
for (int i = 0; i < partInfoList.size(); i++) {
UploadPreRequest.PartInfo partInfo = partInfoList.get(i);
long expires = Long.parseLong(Objects.requireNonNull(Objects.requireNonNull(HttpUrl.parse(partInfo.getUpload_url())).queryParameter("x-oss-expires")));
if (System.currentTimeMillis() / 1000 + 10 >= expires) {
// 已过期重新置换UploadUrl
RefreshUploadUrlRequest refreshUploadUrlRequest = new RefreshUploadUrlRequest();
refreshUploadUrlRequest.setDrive_id(client.getDriveId());
refreshUploadUrlRequest.setUpload_id(uploadPreResult.getUpload_id());
refreshUploadUrlRequest.setFile_id(uploadPreResult.getFile_id());
refreshUploadUrlRequest.setPart_info_list(part_info_list);
String refreshJson = client.post("/file/get_upload_url", refreshUploadUrlRequest);
UploadPreResult refreshResult = JsonUtil.readValue(refreshJson, UploadPreResult.class);
for (int j = i; j < partInfoList.size(); j++) {
UploadPreRequest.PartInfo oldInfo = partInfoList.get(j);
UploadPreRequest.PartInfo newInfo = refreshResult.getPart_info_list().stream().filter(p -> p.getPart_number().equals(oldInfo.getPart_number())).findAny().orElseThrow(NullPointerException::new);
oldInfo.setUpload_url(newInfo.getUpload_url());
}
}
try {
int read = IOUtils.read(inputStream, buffer, 0, buffer.length);
if (read == -1) {
LOGGER.info("文件上传结束。文件名:{},当前进度:{}/{}", path, (i + 1), partInfoList.size());
return;
}
client.upload(partInfo.getUpload_url(), buffer, 0, read);
virtualTFileService.updateLength(parent.getFile_id(), uploadPreResult.getFile_id(), buffer.length);
LOGGER.info("文件正在上传。文件名:{},当前进度:{}/{}", path, (i + 1), partInfoList.size());
LOGGER.info("文件正在上传。文件名:{},当前进度:{}/{}", path, (i+1), partInfoList.size());
} catch (IOException e) {
virtualTFileService.remove(parent.getFile_id(), uploadPreResult.getFile_id());
throw new WebdavException(e);
}
}
@ -202,12 +152,29 @@ public class AliYunDriverClientService {
uploadFinalRequest.setUpload_id(uploadPreResult.getUpload_id());
client.post("/file/complete", uploadFinalRequest);
virtualTFileService.remove(parent.getFile_id(), uploadPreResult.getFile_id());
LOGGER.info("文件上传成功。文件名:{}", path);
// if (!uploadPreResult.getName().equals(pathInfo.getName())) {
// LOGGER.info("上传文件名{}与原文件名{}不同,对文件进行重命名", uploadPreResult.getName(), pathInfo.getName());
// TFile oldFile = getNodeIdByPath2(path);
// // 如果旧文件存在则先删除
// if (oldFile != null) {
// LOGGER.info("旧文件{}还存在,大小为{},进行删除操作,可前往网页版的回收站查看", path, oldFile.getSize());
// remove(path);
// try {
// Thread.sleep(1500);
// } catch (InterruptedException e) {
// // no
// }
// }
// RenameRequest renameRequest = new RenameRequest();
// renameRequest.setDrive_id(client.getDriveId());
// renameRequest.setFile_id(oldFile.getFile_id());
// renameRequest.setName(pathInfo.getName());
// client.post("/file/update", renameRequest);
// }
clearCache();
}
public void rename(String sourcePath, String newName) {
sourcePath = normalizingPath(sourcePath);
TFile tFile = getTFileByPath(sourcePath);
@ -281,15 +248,9 @@ public class AliYunDriverClientService {
return getNodeIdByPath2(path);
}
public Response download(String path, HttpServletRequest request, long size ) {
TFile file = getTFileByPath(path);
DownloadRequest downloadRequest = new DownloadRequest();
downloadRequest.setDrive_id(client.getDriveId());
downloadRequest.setFile_id(file.getFile_id());
String json = client.post("/file/get_download_url", downloadRequest);
Object url = JsonUtil.getJsonNodeValue(json, "url");
LOGGER.debug("{} url = {}", path, url);
return client.download(url.toString(), request, size);
public InputStream download(String path) {
// String downloadUrl = getTFileByPath(path).getDownloadUrl();
return client.download("downloadUrl");
}
private TFile getNodeIdByPath2(String path) {
@ -363,7 +324,11 @@ public class AliYunDriverClientService {
return path;
}
private void clearCache() {
tFilesCache.invalidateAll();
public void clearCache() {
Map<String, Set<TFile>> map = tFilesCache.get();
if (map != null) {
map.clear();
}
}
}

View File

@ -3,12 +3,9 @@ package com.github.zxbu.webdavteambition.store;
import com.github.zxbu.webdavteambition.model.FileType;
import com.github.zxbu.webdavteambition.model.PathInfo;
import com.github.zxbu.webdavteambition.model.result.TFile;
import net.sf.webdav.ITransaction;
import net.sf.webdav.IWebdavStore;
import net.sf.webdav.StoredObject;
import net.sf.webdav.Transaction;
import net.sf.webdav.*;
import net.sf.webdav.exceptions.UnauthenticatedException;
import net.sf.webdav.exceptions.WebdavException;
import okhttp3.Response;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -19,7 +16,6 @@ import java.io.IOException;
import java.io.InputStream;
import java.security.Principal;
import java.util.Enumeration;
import java.util.Optional;
import java.util.Set;
public class AliYunDriverFileSystemStore implements IWebdavStore {
@ -39,23 +35,28 @@ public class AliYunDriverFileSystemStore implements IWebdavStore {
@Override
public void destroy() {
LOGGER.info("destroy");
LOGGER.debug("destroy");
}
@Override
public ITransaction begin(Principal principal, HttpServletRequest req, HttpServletResponse resp) {
LOGGER.debug("begin");
aliYunDriverClientService.clearCache();
return new Transaction(principal, req, resp);
}
@Override
public void checkAuthentication(ITransaction transaction) {
LOGGER.debug("checkAuthentication");
if (transaction.getPrincipal() == null) {
throw new UnauthenticatedException(WebdavStatus.SC_UNAUTHORIZED);
}
}
@Override
public void commit(ITransaction transaction) {
aliYunDriverClientService.clearCache();
LOGGER.debug("commit");
}
@ -80,24 +81,8 @@ public class AliYunDriverFileSystemStore implements IWebdavStore {
@Override
public InputStream getResourceContent(ITransaction transaction, String resourceUri) {
LOGGER.info("getResourceContent: {}", resourceUri);
Enumeration<String> headerNames = transaction.getRequest().getHeaderNames();
while (headerNames.hasMoreElements()) {
String s = headerNames.nextElement();
LOGGER.debug("{} request: {} = {}",resourceUri, s, transaction.getRequest().getHeader(s));
}
HttpServletResponse response = transaction.getResponse();
long size = getResourceLength(transaction, resourceUri);
Response downResponse = aliYunDriverClientService.download(resourceUri, transaction.getRequest(), size);
response.setContentLengthLong(downResponse.body().contentLength());
LOGGER.debug("{} code = {}", resourceUri, downResponse.code());
for (String name : downResponse.headers().names()) {
LOGGER.debug("{} downResponse: {} = {}", resourceUri, name, downResponse.header(name));
response.addHeader(name, downResponse.header(name));
}
response.setStatus(downResponse.code());
return downResponse.body().byteStream();
LOGGER.debug("getResourceContent: {}", resourceUri);
return aliYunDriverClientService.download(resourceUri);
}
@Override
@ -108,8 +93,7 @@ public class AliYunDriverFileSystemStore implements IWebdavStore {
long contentLength = request.getContentLength();
if (contentLength < 0) {
contentLength = Long.parseLong(Optional.ofNullable(request.getHeader("content-length"))
.orElse(request.getHeader("X-Expected-Entity-Length")));
contentLength = Long.parseLong(request.getHeader("content-length"));
}
aliYunDriverClientService.uploadPre(resourceUri, contentLength, content);
@ -131,7 +115,7 @@ public class AliYunDriverFileSystemStore implements IWebdavStore {
@Override
public String[] getChildrenNames(ITransaction transaction, String folderUri) {
LOGGER.info("getChildrenNames: {}", folderUri);
LOGGER.debug("getChildrenNames: {}", folderUri);
TFile tFile = aliYunDriverClientService.getTFileByPath(folderUri);
if (tFile.getType().equals(FileType.file.name())) {
return new String[0];
@ -144,11 +128,7 @@ public class AliYunDriverFileSystemStore implements IWebdavStore {
@Override
public long getResourceLength(ITransaction transaction, String path) {
return getResourceLength2(transaction, path);
}
public long getResourceLength2(ITransaction transaction, String path) {
LOGGER.info("getResourceLength: {}", path);
LOGGER.debug("getResourceLength: {}", path);
TFile tFile = aliYunDriverClientService.getTFileByPath(path);
if (tFile == null || tFile.getSize() == null) {
return 384;
@ -186,12 +166,12 @@ public class AliYunDriverFileSystemStore implements IWebdavStore {
public StoredObject getStoredObject(ITransaction transaction, String uri) {
LOGGER.info("getStoredObject: {}", uri);
LOGGER.debug("getStoredObject: {}", uri);
TFile tFile = aliYunDriverClientService.getTFileByPath(uri);
if (tFile != null) {
StoredObject so = new StoredObject();
so.setFolder(tFile.getType().equalsIgnoreCase("folder"));
so.setResourceLength(getResourceLength2(transaction, uri));
so.setResourceLength(getResourceLength(transaction, uri));
so.setCreationDate(tFile.getCreated_at());
so.setLastModified(tFile.getUpdated_at());
return so;

View File

@ -1,68 +0,0 @@
package com.github.zxbu.webdavteambition.store;
import com.github.zxbu.webdavteambition.model.FileType;
import com.github.zxbu.webdavteambition.model.result.TFile;
import com.github.zxbu.webdavteambition.model.result.UploadPreResult;
import org.springframework.stereotype.Service;
import java.util.Collection;
import java.util.Collections;
import java.util.Date;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
/**
* 虚拟文件用于上传时列表展示
*/
@Service
public class VirtualTFileService {
private final Map<String, Map<String, TFile>> virtualTFileMap = new ConcurrentHashMap<>();
/**
* 创建文件
*/
public void createTFile(String parentId, UploadPreResult uploadPreResult) {
Map<String, TFile> tFileMap = virtualTFileMap.computeIfAbsent(parentId, s -> new ConcurrentHashMap<>());
tFileMap.put(uploadPreResult.getFile_id(), convert(uploadPreResult));
}
public void updateLength(String parentId, String fileId, long length) {
Map<String, TFile> tFileMap = virtualTFileMap.get(parentId);
if (tFileMap == null) {
return;
}
TFile tFile = tFileMap.get(fileId);
if (tFile == null) {
return;
}
tFile.setSize(tFile.getSize() + length);
tFile.setUpdated_at(new Date());
}
public void remove(String parentId, String fileId) {
Map<String, TFile> tFileMap = virtualTFileMap.get(parentId);
if (tFileMap == null) {
return;
}
tFileMap.remove(fileId);
}
public Collection<TFile> list(String parentId) {
Map<String, TFile> tFileMap = virtualTFileMap.get(parentId);
if (tFileMap == null) {
return Collections.emptyList();
}
return tFileMap.values();
}
private TFile convert(UploadPreResult uploadPreResult) {
TFile tFile = new TFile();
tFile.setCreated_at(new Date());
tFile.setFile_id(uploadPreResult.getFile_id());
tFile.setName(uploadPreResult.getFile_name());
tFile.setType(FileType.file.name());
tFile.setUpdated_at(new Date());
tFile.setSize(0L);
return tFile;
}
}

View File

@ -18,7 +18,6 @@ package net.sf.webdav.methods;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Arrays;
@ -33,8 +32,6 @@ import net.sf.webdav.IWebdavStore;
import net.sf.webdav.StoredObject;
import net.sf.webdav.WebdavStatus;
import net.sf.webdav.locking.ResourceLocks;
import org.apache.tomcat.util.http.fileupload.IOUtils;
import org.springframework.web.util.UriUtils;
public class DoGet extends DoHead {
@ -64,33 +61,30 @@ public class DoGet extends DoHead {
OutputStream out = resp.getOutputStream();
InputStream in = _store.getResourceContent(transaction, path);
try {
if (in != null) {
LOG.debug("开始 {}, ", path);
IOUtils.copyLarge(in, out);
LOG.debug("结束 {}", path);
int read = -1;
byte[] copyBuffer = new byte[BUF_SIZE];
while ((read = in.read(copyBuffer, 0, copyBuffer.length)) != -1) {
out.write(copyBuffer, 0, read);
}
} finally {
// flushing causes a IOE if a file is opened on the webserver
// client disconnected before server finished sending response
try {
if (in != null) {
in.close();
}
in.close();
} catch (Exception e) {
LOG.warn("{} Closing InputStream causes Exception!\n", path
,e);
LOG.warn("Closing InputStream causes Exception!\n"
+ e.toString());
}
try {
out.flush();
out.close();
} catch (Exception e) {
LOG.warn("{} Flushing OutputStream causes Exception!\n", path
,e);
LOG.warn("Flushing OutputStream causes Exception!\n"
+ e.toString());
}
}
} catch (Exception e) {
LOG.warn("{} doBody causes Exception!\n", path
,e);
LOG.trace(e.toString());
}
}
@ -147,7 +141,7 @@ public class DoGet extends DoHead {
childrenTemp.append("\">");
childrenTemp.append("<td>");
childrenTemp.append("<a href=\"");
childrenTemp.append(UriUtils.encode(child, "utf-8"));
childrenTemp.append(child);
StoredObject obj= _store.getStoredObject(transaction, path+"/"+child);
if (obj == null)
{
@ -210,7 +204,7 @@ public class DoGet extends DoHead {
/**
* Return the CSS styles used to display the HTML representation
* of the webdav content.
*
*
* @return
*/
protected String getCSS()
@ -272,7 +266,7 @@ public class DoGet extends DoHead {
/**
* Return the header to be displayed in front of the folder content
*
*
* @param transaction
* @param path
* @param resp
@ -287,7 +281,7 @@ public class DoGet extends DoHead {
/**
* Return the footer to be displayed after the folder content
*
*
* @param transaction
* @param path
* @param resp

View File

@ -209,9 +209,6 @@ public class DoLock extends AbstractMethod {
return;
}
nullSo = _store.getStoredObject(transaction, _path);
if (nullSo == null) {
nullSo = new StoredObject();
}
// define the newly created resource as null-resource
nullSo.setNullResource(true);
@ -419,13 +416,8 @@ public class DoLock extends AbstractMethod {
if (currentNode.getNodeType() == Node.ELEMENT_NODE
|| currentNode.getNodeType() == Node.TEXT_NODE) {
if (currentNode.getFirstChild() != null) {
_lockOwner = currentNode.getFirstChild()
.getNodeValue();
} else {
_lockOwner = currentNode.toString();
}
_lockOwner = currentNode.getFirstChild()
.getNodeValue();
}
}
}

View File

@ -72,11 +72,6 @@ public class DoUnlock extends DeterminableMethod {
if (_resourceLocks.unlock(transaction, lockId, owner)) {
StoredObject so = _store.getStoredObject(
transaction, path);
if (so == null) {
resp.sendError(HttpServletResponse.SC_NOT_FOUND);
return;
}
if (so.isNullResource()) {
_store.removeObject(transaction, path);
}

View File

@ -5,5 +5,3 @@ logging.file.name=webdav.log
aliyundrive.auth.enable=true
aliyundrive.auth.user-name=admin
aliyundrive.auth.password=admin
server.tomcat.connection-timeout=1d