Restructure storage filesystem directory #13300

rajkan01 · 2020-07-15T17:56:51Z

Summary of changes

Restructured storage/filesystem as per the new directory structure proposal:

storage
├── blockdevice
├── filesystem
│   ├── mbed_lib.json
│   ├── include
│   │   └── filesystem
│   │       ├── mbed_filesystem.h
│   │       ├── Dir.h
│   │       └── [...]
│   ├── source
│   │   ├── Dir.cpp
│   │   └── [...]
│   ├── tests
│   ├── fat
│   │   ├── include
│   │   │   └── fat
│   │   ├── source
│   │   ├── tests
│   │   └── [...]
│   ├── littlefs
│   │   └── [...]
│   └── littlefsv2
│       └── [...]

Impact of changes

None.

Migration actions required

None.

Documentation

To Be Updated.

Pull request type

[x] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
[] Feature update (New feature / Functionality change / New API)
[] Major update (Breaking change E.g. Return code change / API behaviour change)

Test results

[] No Tests required for this change (E.g docs only update)
[x] Covered by existing mbed-os tests (Greentea or Unittest)
[x] Tests / results supplied as part of this PR

Manual testing: (Build for K64F target with GCC_ARM toolchain)

mbed-os-example-blockdevice
mbed-os-example-filesystem
Greentea tests: Full and Bare metal profiles
Unit tests build and run

Reviewers

@0xc0170 @ARMmbed/mbed-os-core

ciarmcom · 2020-07-15T19:00:36Z

@rajkan01, thank you for your changes.
@0xc0170 @ARMmbed/mbed-os-core @ARMmbed/mbed-os-storage @ARMmbed/mbed-os-maintainers please review.

LDong-Arm

Thanks @rajkan01, the directory structure looks good to me in general. Please fix the line endings.

@evedon I've raised one questions regarding the "sub" libraries (fat, littlefs, etc.) under filesystem.

LDong-Arm · 2020-07-16T14:48:28Z

storage/filesystem/littlefsv2/tests/TESTS/util/Makefile

-
-clean:
-	./clean.sh
+


The line ending has been changed from \n to \r\n (Windows) - this is why we see a big diff on GitHub. Please revert it back - it'll make review (and git blame in the future) easier.

LDong-Arm · 2020-07-16T14:49:54Z

storage/filesystem/littlefsv2/tests/TESTS/util/clean.sh

-
-rm -f main.cpp
-rm -f template_all_names.txt
+#!/bin/bash


Line ending

LDong-Arm · 2020-07-16T14:50:05Z

storage/filesystem/littlefsv2/tests/TESTS/util/replacements_mbed.yml

- ['info\.name', 'ent.d_name']
- ['info\.type', 'ent.d_type']
- ['^.*info\.size.*$', ''] # dirent sizes not supported
+- ['lfs2_format\(&lfs2, &cfg\)', 'MBED_TEST_FILESYSTEM::format(&bd)']


Line ending

LDong-Arm · 2020-07-16T14:50:15Z

storage/filesystem/littlefsv2/tests/TESTS/util/replacements_retarget.yml

- ['LFS2_O_WRONLY \| LFS2_O_CREAT', '"wb"']
- ['LFS2_O_RDONLY', '"rb"']
- ['LFS2_O_RDWR', '"r+b"']
+- ['lfs2_format\(&lfs2, &cfg\)', 'MBED_TEST_FILESYSTEM::format(&bd)']


Line ending

LDong-Arm · 2020-07-16T14:50:22Z

storage/filesystem/littlefsv2/tests/TESTS/util/template_subunit.fmt

-    {{
-{test}
-    }}
+


Line ending

LDong-Arm · 2020-07-16T14:50:26Z

storage/filesystem/littlefsv2/tests/TESTS/util/template_unit.fmt

-    res = bd.deinit();
-    TEST_ASSERT_EQUAL(0, res);
-}}
+


Line ending

LDong-Arm · 2020-07-16T14:50:30Z

storage/filesystem/littlefsv2/tests/TESTS/util/template_wrapper.fmt

-int main() {{
-    return !Harness::run(specification);
-}}
+#include "mbed.h"


Line ending

LDong-Arm · 2020-07-16T15:01:24Z

storage/system_storage/SystemStorage.cpp

+#include "filesystem/FileSystem.h"
+#include "fat/FATFileSystem.h"
+#include "littlefs/LittleFileSystem.h"


Something I'd like to discuss:
In terms of user experience, including "filesystem/FATFileSystem.h", "filesystem/LittleFileSystem.h" in code seems more sensible than "fat/FATFileSystem.h", "littlefs/LittleFileSystem.h". If we want to achieve that, we can have

storage ├── filesystem │ ├── fat │ │ └── include │ │ └── filesystem # changed from fat │ │ └── FATFileSystem.h

Of course, this would be a slight change from our initial proposal. What do you think? @rajkan01 @evedon

It breaks the consistency in the directory structure for little benefit. I don't think it is worth doing.

evedon · 2020-07-16T16:36:58Z

File permission on storage/filesystem/littlefs/littlefs/tests/template.fmt should be reverted to 644. Apart from line endings issues noted by @LDong-Arm, code change is fine.

LDong-Arm

LGTM

mergify · 2020-07-17T09:46:30Z

This PR cannot be merged due to conflicts. Please rebase to resolve them.

0xc0170 · 2020-07-20T11:34:31Z

CI started

mbed-ci · 2020-07-20T12:36:20Z

Test run: SUCCESS

Summary: 6 of 6 test jobs passed
Build number : 1
Build artifacts

rajkan01 · 2020-07-20T13:38:28Z

@0xc0170 This PR is ready for merge. Could you merge...

0xc0170 · 2020-07-20T14:05:56Z

One last check : changing permissions on some scripts within this PR was intentional ?

See for instance littlefs/tests/TESTS/util/clean.sh 100644 → 100755

rajkan01 · 2020-07-20T14:14:58Z

One last check : changing permissions on some scripts within this PR was intentional ?
See for instance littlefs/tests/TESTS/util/clean.sh 100644 → 100755

@0xc0170 Revert to the previous permission as this was changed unintentionally on the last PR #13244 and even you have commented #13244 (comment)

rajkan01 force-pushed the storage_fsdir_restructure branch 2 times, most recently from 0ef689c to eca9b3b Compare July 15, 2020 18:10

rajkan01 changed the title ~~Storage fsdir restructure~~ Restructure storage filesystem directory Jul 15, 2020

ciarmcom added the release-type: patch Indentifies a PR as containing just a patch label Jul 15, 2020

ciarmcom requested review from 0xc0170 and a team July 15, 2020 19:00

ciarmcom added the needs: review label Jul 15, 2020

LDong-Arm suggested changes Jul 16, 2020

View reviewed changes

rajkan01 force-pushed the storage_fsdir_restructure branch 2 times, most recently from c2d2b6e to ef5602f Compare July 16, 2020 17:21

LDong-Arm approved these changes Jul 17, 2020

View reviewed changes

mergify bot added needs: work and removed needs: review labels Jul 17, 2020

rajkan01 force-pushed the storage_fsdir_restructure branch from e0eb4e2 to 699acaa Compare July 17, 2020 19:38

rajkan01 added 3 commits July 17, 2020 12:42

Refactor storage/filesystem directory

b340492

Move greentea test closure to library

41f0796

Update header path reference

51247d0

rajkan01 force-pushed the storage_fsdir_restructure branch from 699acaa to 51247d0 Compare July 17, 2020 19:43

rajkan01 mentioned this pull request Jul 17, 2020

Restructure storage kvstore directory #13307

Merged

evedon approved these changes Jul 20, 2020

View reviewed changes

mergify bot added needs: CI and removed needs: work labels Jul 20, 2020

rajkan01 mentioned this pull request Jul 20, 2020

Storage system_storage directory restructure #13314

Merged

mergify bot added ready for merge and removed needs: CI labels Jul 20, 2020

0xc0170 approved these changes Jul 20, 2020

View reviewed changes

0xc0170 merged commit 3a8989a into ARMmbed:master Jul 20, 2020

mergify bot removed the ready for merge label Jul 20, 2020

mbedmain added release-version: 6.2.1 Release-pending and removed release-type: patch Indentifies a PR as containing just a patch Release-pending labels Aug 16, 2020


		clean:
		./clean.sh

Restructure storage filesystem directory #13300

Restructure storage filesystem directory #13300

Uh oh!

Conversation

rajkan01 commented Jul 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Impact of changes

Migration actions required

Documentation

Pull request type

Test results

Reviewers

Uh oh!

ciarmcom commented Jul 15, 2020

Uh oh!

LDong-Arm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evedon commented Jul 16, 2020

Uh oh!

LDong-Arm left a comment

Choose a reason for hiding this comment

Uh oh!

mergify bot commented Jul 17, 2020

Uh oh!

0xc0170 commented Jul 20, 2020

Uh oh!

mbed-ci commented Jul 20, 2020

Uh oh!

rajkan01 commented Jul 20, 2020

Uh oh!

0xc0170 commented Jul 20, 2020

Uh oh!

rajkan01 commented Jul 20, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

rajkan01 commented Jul 15, 2020 •

edited

Loading

rajkan01 commented Jul 20, 2020 •

edited

Loading